Background schedule simulations in an intelligent, network-connected thermostat

ABSTRACT

Various arrangements for promoting energy efficiency in association with an HVAC system of an enclosure are presented. A first HVAC schedule may be accessed that includes setpoints. The HVAC system may be operated according to the first HVAC schedule. The first HVAC schedule may be processed to generate a second HVAC schedule representative of what would have been generated by an automated schedule learning algorithm operating over the period of time. The second HVAC schedule can be simulated using a thermal model of the enclosure to determine a hypothetical cost of operating the HVAC system according to the second HVAC schedule over the period of time. Information representative of an energy cost difference between an actual cost of operating the HVAC system according to the first HVAC schedule and the hypothetical cost of operating the HVAC system according to the second HVAC schedule can be generated.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. Ser. No. 13/828,767, which isa continuation-in-part of International Application No.PCT/US2012/000007 filed Jan. 3, 2012 and of U.S. Ser. No. 13/632,041filed Sep. 30, 2012. International Application No. PCT/US2012/000007claims the benefit of U.S. Provisional Application No. 61/429,093 filedDec. 31, 2010 and U.S. Provisional Application No. 61/627,996 filed Oct.21, 2011. U.S. Ser. No. 13/632,041 claims the benefit of U.S.Provisional Application No. 61/550,346 filed Oct. 21, 2011. Each of theabove-referenced patent applications is incorporated by referenceherein.

TECHNICAL FIELD

The current patent application is directed to machine learning andintelligent controllers and, in particular, to intelligent controllers,and machine-learning methods incorporated within intelligentcontrollers, that develop and refine one or more control schedules, overtime, based on received control inputs and inputs throughschedule-creation and schedule-modification interfaces.

BACKGROUND

Control systems and control theory are well-developed fields of researchand development that have had a profound impact on the design anddevelopment of a large number of systems and technologies, fromairplanes, spacecraft, and other vehicle and transportation systems tocomputer systems, industrial manufacturing and operations facilities,machine tools, process machinery, and consumer devices. Control theoryencompasses a large body of practical, system-control-design principles,but is also an important branch of theoretical and applied mathematics.Various different types of controllers are commonly employed in manydifferent application domains, from simple closed-loop feedbackcontrollers to complex, adaptive, state-space anddifferential-equations-based processor-controlled control system.

Many controllers are designed to output control signals to variousdynamical components of a system based on a control model and sensorfeedback from the system. Many systems are designed to exhibit apredetermined behavior or mode of operation, and the control componentsof the system are therefore designed, by traditional design andoptimization techniques, to ensure that the predetermined systembehavior transpires under normal operational conditions. A moredifficult control problem involves design and implementation ofcontrollers that can produce desired system operational behaviors thatare specified following controller design and implementation.Theoreticians, researchers, and developers of many different types ofcontrollers and automated systems continue to seek approaches tocontroller design to produce controllers with the flexibility andintelligence to control systems to produce a wide variety of differentoperational behaviors, including operational behaviors specified aftercontroller design and manufacture.

SUMMARY

The current application is directed to intelligent controllers thatinitially aggressively learn, and then continue, in a steady-state mode,to monitor, learn, and modify one or more control schedules that specifya desired operational behavior of a device, machine, system, ororganization controlled by the intelligent controller. An intelligentcontroller generally acquires one or more initial control schedulesthrough schedule-creation and schedule-modification interfaces or byaccessing a default control schedule stored locally or remotely in amemory or mass-storage device. The intelligent controller then proceedsto learn, over time, a desired operational behavior for the device,machine, system, or organization controlled by the intelligentcontroller based on immediate-control inputs, schedule-modificationinputs, and previous and current control schedules, encoding the desiredoperational behavior in one or more control schedules and/orsub-schedules. Control-schedule learning is carried out in at least twodifferent phases, the first of which favors frequent immediate-controlinputs and the remaining of which generally tend to minimize thefrequency of immediate-control inputs. Learning occurs followingmonitoring periods and learning results may be propagated from a newprovisional control schedule or sub-schedule associated with a completedmonitoring period to one or more related control schedules orsub-schedules associated with related time periods.

Provided according to some embodiments is a method for promoting energyefficiency in association with an HVAC system of a climate controlledenclosure, the HVAC system being controlled by a network-connectedthermostat having a user interface controllable by a user. The methodincludes receiving a first HVAC schedule including a plurality ofsetpoints that each include an indication of a setpoint time and asetpoint temperature for the climate controlled enclosure, operating,over a period of time, the HVAC system according to the first HVACschedule, measuring, over the period of time, an actual ambienttemperature in the climate controlled enclosure, receiving an updatecorresponding to at least one of: a real-time setpoint change includinga desired temperature for immediate implementation entered by the userusing the user interface during the period of time, a schedule changeincluding at least one of (i) a change to the setpoint temperature orsetpoint time of one or more setpoints of the first HVAC schedule, (ii)the removal of the one or more setpoints from the first HVAC schedule,and (iii) the addition of one or more new setpoints to the first HVACschedule, and an occupancy profile including indications over the periodof time of whether one or more occupants is present in the climatecontrolled enclosure, processing the received update in conjunction withthe first HVAC schedule to generate a second HVAC schedulerepresentative of what would have been generated by a first automatedschedule learning algorithm operating over the period of time,generating a cost difference indicator, subsequent to the period oftime, corresponding to a cost difference between an actual cost ofoperating the HVAC system according to the first HVAC schedule over theperiod of time and a hypothetical cost of operating the HVAC systemaccording to the second HVAC schedule over the period of time, anddisplaying the cost difference indicator to the user using the userinterface.

Provided according to some embodiments is a method for promoting energyefficiency in association with an HVAC system of a climate controlledenclosure, the HVAC system being controlled by a network-connectedthermostat having a user interface controllable by a user. The methodincludes measuring an actual ambient temperature in a climate controlledenclosure over a period of time, generating a plurality of controlsignals, to direct the operation of an HVAC system, over the period oftime according to at least one of a first HVAC schedule including aplurality of setpoints, each setpoint including an indication of asetpoint time and a desired setpoint temperature for the climatecontrolled enclosure, and an update includes at least one of a real-timesetpoint change, a schedule change, and an occupancy profile,determining the actual energy consumed by operating the HVAC systemaccording to the plurality of control signals over the period of time,receiving data indicative of the ambient temperature of an area outsidethe climate controlled enclosure, processing the actual ambienttemperature in the enclosure over the period of time in conjunction withthe outside temperature over the period of time and the control signalsto generate a thermal model of the climate controlled enclosure,generating a hypothetical runtime profile from the first HVAC schedule,the period of time, the update, the thermal model of the climatecontrolled enclosure, the actual ambient temperature in the climatecontrolled enclosure, and the data indicative of the temperature of theambient temperature of the area outside the climate controlledenclosure, calculating the hypothetical energy consumed by the HVACsystem as a result of the hypothetical runtime profile during the periodof time, generating an indicator of the difference between thehypothetical energy consumed by the HVAC system and the actual energyconsumed by the HVAC system for the period of time, and providing theindicator of the difference to a user.

Provided according to some embodiments is a system for promoting energyefficiency in association with an HVAC system of a climate controlledenclosure, the HVAC system being controlled by a network-connectedthermostat having a user interface controllable by a user. The systemincludes a sensor that measures ambient temperature in a climatecontrolled enclosure, and a processor that requests measurement of anactual ambient temperature in a climate controlled enclosure over aperiod of time, generates a plurality of control signals, to direct theoperation of an HVAC system, over the period of time according to atleast one of a first HVAC schedule including a plurality of setpointsthat each include an indication of a setpoint time and a desiredsetpoint temperature for the climate controlled enclosure, and an updatethat includes at least one of a real-time setpoint change, a schedulechange, and an occupancy profile, determines the actual energy consumedby operating the HVAC system according to the plurality of controlsignals over the period of time, receives data indicative of the ambienttemperature of an area outside the climate controlled enclosure,processes the actual ambient temperature in the enclosure over theperiod of time in conjunction with the outside temperature over theperiod of time and the control signals to generate a thermal model ofthe climate controlled enclosure, generates a hypothetical runtimeprofile from the first HVAC schedule, the period of time, the update,the thermal model of the climate controlled enclosure, the actualambient temperature in the climate controlled enclosure, and the dataindicative of the temperature of the ambient temperature of the areaoutside the climate controlled enclosure, calculates the hypotheticalenergy consumed by the HVAC system as a result of the hypotheticalruntime profile during the period of time, generates an indicator of thedifference between the hypothetical energy consumed by the HVAC systemand the actual energy consumed by the HVAC system for the period oftime, and provides the indicator of the difference to a user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a smart-home environment.

FIG. 2 illustrates integration of smart-home devices with remote devicesand systems.

FIG. 3 illustrates information processing within the environment ofintercommunicating entities illustrated in FIG. 2.

FIG. 4 illustrates a general class of intelligent controllers to whichthe current application is directed.

FIG. 5 illustrates additional internal features of an intelligentcontroller.

FIG. 6 illustrates a generalized computer architecture that representsan example of the type of computing machinery that may be included in anintelligent controller, server computer, and other processor-basedintelligent devices and systems.

FIG. 7 illustrates features and characteristics of an intelligentcontroller of the general class of intelligent controllers to which thecurrent application is directed.

FIG. 8 illustrates a typical control environment within which anintelligent controller operates.

FIG. 9 illustrates the general characteristics of sensor output.

FIGS. 10A-D illustrate information processed and generated by anintelligent controller during control operations.

FIGS. 11A-E provide a transition-state-diagram-based illustration ofintelligent-controller operation.

FIG. 12 provides a state-transition diagram that illustrates automatedcontrol-schedule learning.

FIG. 13 illustrates time frames associated with an example controlschedule that includes shorter-time-frame sub-schedules.

FIGS. 14A-C show three different types of control schedules.

FIGS. 15A-G show representations of immediate-control inputs that may bereceived and executed by an intelligent controller, and then recordedand overlaid onto control schedules, such as those discussed above withreference to FIGS. 14A-C, as part of automated control-schedulelearning.

FIGS. 16A-E illustrate one aspect of the method by which a new controlschedule is synthesized from an existing control schedule and recordedschedule changes and immediate-control inputs.

FIGS. 17A-E illustrate one approach to resolving schedule clusters.

FIGS. 18A-B illustrate the effect of a prospective schedule changeentered by a user during a monitoring period.

FIGS. 19A-B illustrate the effect of a retrospective schedule changeentered by a user during a monitoring period.

FIGS. 20A-C illustrate overlay of recorded data onto an existing controlschedule, following completion of a monitoring period, followed byclustering and resolution of clusters.

FIGS. 21A-B illustrate the setpoint-spreading operation.

FIGS. 22A-B illustrate schedule propagation.

FIGS. 23A-C illustrate new-provisional-schedule propagation usingP-value vs. t control-schedule plots.

FIGS. 24A-I illustrate a number of example rules used to simplify apre-existing control schedule overlaid with propagated setpoints as partof the process of generating a new provisional schedule.

FIGS. 25A-M illustrate an example implementation of an intelligentcontroller that incorporates the above-describedautomated-control-schedule-learning method.

FIG. 26 illustrates three different week-based control schedulescorresponding to three different control modes for operation of anintelligent controller.

FIG. 27 illustrates a state-transition diagram for an intelligentcontroller that operates according to seven different control schedules.

FIGS. 28A-C illustrate one type of control-schedule transition that maybe carried out by an intelligent controller.

FIGS. 29-30 illustrate types of considerations that may be made by anintelligent controller during steady-state-learning phases.

FIG. 31A illustrates a perspective view of an intelligent thermostat.

FIGS. 31B-31C illustrate the intelligent thermostat being controlled bya user.

FIG. 32 illustrates an exploded perspective view of the intelligentthermostat and an HVAC-coupling wall dock.

FIGS. 33A-33B illustrate exploded front and rear perspective views ofthe intelligent thermostat.

FIGS. 34A-34B illustrate exploded front and rear perspective views,respectively, of the head unit.

FIGS. 35A-35B illustrate exploded front and rear perspective views,respectively, of the head unit frontal assembly.

FIGS. 36A-36B illustrate exploded front and rear perspective views,respectively, of the backplate unit.

FIG. 37 shows a perspective view of a partially assembled head unit.

FIG. 38 illustrates the head unit circuit board.

FIG. 39 illustrates a rear view of the backplate circuit board.

FIGS. 40A-D-2 illustrate steps for achieving initial learning.

FIGS. 41A-M illustrate a progression of conceptual views of a thermostatcontrol schedule.

FIGS. 42A-42B illustrate steps for steady-state learning.

FIG. 43A illustrates the intelligent thermostat as installed in a smarthome environment having an HVAC system and a set of control wiresextending therefrom.

FIG. 43B illustrates an exemplary diagram of the HVAC system of FIG.43A.

FIG. 44 illustrates components of a background schedule simulationsystem.

FIG. 45 illustrates an exemplary implementation of the backgroundschedule simulation system.

FIGS. 46 to 48 illustrate examples of remote graphical user interfacedisplays presented to the user on their data appliance for managingtheir background schedule simulation system and/or otherwise interactingwith their smart home environment including the background schedulesimulation system.

DETAILED DESCRIPTION

The current application is directed to a general class of intelligentcontrollers that includes many different specific types of intelligentcontrollers that can be applied to, and incorporated within, manydifferent types of devices, machines, systems, and organizations.Intelligent controllers control the operation of devices, machines,systems, and organizations. The general class of intelligent controllersto which the current application is directed include automated learningcomponents that allow the intelligent controllers to learn desiredoperational behaviors of the devices, machines, systems, andorganizations which they control and incorporate the learned informationinto control schedules. The subject matter of this patent specificationrelates to the subject matter of the following commonly assignedapplications, each of which is incorporated by reference herein: U.S.Ser. No. 12/881,463 filed Sep. 14, 2010; U.S. Ser. No. 13/269,501 filedOct. 7, 2011; U.S. Ser. No. 13/317,423 filed Oct. 17, 2011; U.S. Ser.No. 13/440,910 filed Apr. 5, 2012; U.S. Ser. No. 13/632,028 filed Sep.30, 2012; and U.S. Ser. No. 13/632,070 filed Sep. 30, 2012.

The current application discloses, in addition to methods andimplementations for automated control-schedule learning, a specificexample of an intelligent thermostat controller, or intelligentthermostat, and a specific control-schedule-learning method for theintelligent thermostat that serves as a detailed example of theautomated control-schedule-learning methods employed by the generalclass of intelligent controllers to which the current application isdirected. The intelligent thermostat is an example of a smart-homedevice.

The detailed description includes four subsections: (1) an overview ofthe smart-home environment; (2) automated control-schedule learning; (3)automated control-schedule learning in the context of an intelligentthermostat; and (4) background schedule simulation. The first subsectionprovides a description of one area of technology that offers manyopportunities for application and incorporation ofautomated-control-schedule-learning methods. The second subsectionprovides a detailed description of automated control-schedule learning,including a first, general implementation. The third subsection providesa specific example of an automated-control-schedule-learning methodincorporated within an intelligent thermostat. The fourth subsectionprovides a detailed description of background schedule simulation,including systems and processes for background schedule simulation

Overview of the Smart-Home Environment

FIG. 1 illustrates a smart-home environment. The smart-home environment100 includes a number of intelligent, multi-sensing, network-connecteddevices. These smart-home devices intercommunicate and are integratedtogether within the smart-home environment. The smart-home devices mayalso communicate with cloud-based smart-home control and/ordata-processing systems in order to distribute control functionality, toaccess higher-capacity and more reliable computational facilities, andto integrate a particular smart home into a larger, multi-home orgeographical smart-home-device-based aggregation.

The smart-home devices may include one more intelligent thermostats 102,one or more intelligent hazard-detection units 104, one or moreintelligent entryway-interface devices 106, smart switches, includingsmart wall-like switches 108, smart utilities interfaces and otherservices interfaces, such as smart wall-plug interfaces 110, and a widevariety of intelligent, multi-sensing, network-connected appliances 112,including refrigerators, televisions, washers, dryers, lights, audiosystems, intercom systems, mechanical actuators, wall air conditioners,pool-heating units, irrigation systems, and many other types ofintelligent appliances and systems.

In general, smart-home devices include one or more different types ofsensors, one or more controllers and/or actuators, and one or morecommunications interfaces that connect the smart-home devices to othersmart-home devices, routers, bridges, and hubs within a local smart-homeenvironment, various different types of local computer systems, and tothe Internet, through which a smart-home device may communicate withcloud-computing servers and other remote computing systems. Datacommunications are generally carried out using any of a large variety ofdifferent types of communications media and protocols, includingwireless protocols, such as Wi-Fi, ZigBee, 6LoWPAN, various types ofwired protocols, including CAT6 Ethernet, HomePlug, and other such wiredprotocols, and various other types of communications protocols andtechnologies. Smart-home devices may themselves operate as intermediatecommunications devices, such as repeaters, for other smart-home devices.The smart-home environment may additionally include a variety ofdifferent types of legacy appliances and devices 140 and 142 which lackcommunications interfaces and processor-based controllers.

FIG. 2 illustrates integration of smart-home devices with remote devicesand systems. Smart-home devices within a smart-home environment 200 cancommunicate through the Internet 202 via 3G/4G wireless communications204, through a hubbed network 206, or by other communications interfacesand protocols. Many different types of smart-home-related data, and dataderived from smart-home data 208, can be stored in, and retrieved from,a remote system 210, including a cloud-based remote system. The remotesystem may include various types of statistics, inference, and indexingengines 212 for data processing and derivation of additional informationand rules related to the smart-home environment. The stored data can beexposed, via one or more communications media and protocols, in part orin whole, to various remote systems and organizations, includingcharities 214, governments 216, academic institutions 218, businesses220, and utilities 222. In general, the remote data-processing system210 is managed or operated by an organization or vendor related tosmart-home devices or contracted for remote data-processing and otherservices by a homeowner, landlord, dweller, or othersmart-home-associated user. The data may also be further processed byadditional commercial-entity data-processing systems 213 on behalf ofthe smart-homeowner or manager and/or the commercial entity or vendorwhich operates the remote data-processing system 210. Thus, externalentities may collect, process, and expose information collected bysmart-home devices within a smart-home environment, may process theinformation to produce various types of derived results which may becommunicated to, and shared with, other remote entities, and mayparticipate in monitoring and control of smart-home devices within thesmart-home environment as well as monitoring and control of thesmart-home environment. Of course, in many cases, export of informationfrom within the smart-home environment to remote entities may bestrictly controlled and constrained, using encryption, access rights,authentication, and other well-known techniques, to ensure thatinformation deemed confidential by the smart-home manager and/or by theremote data-processing system is not intentionally or unintentionallymade available to additional external computing facilities, entities,organizations, and individuals.

FIG. 3 illustrates information processing within the environment ofintercommunicating entities illustrated in FIG. 2. The variousprocessing engines 212 within the external data-processing system 210can process data with respect to a variety of different goals, includingprovision of managed services 302, various types of advertizing andcommunications 304, social-networking exchanges and other electronicsocial communications 306, and for various types of monitoring andrule-generation activities 308. The various processing engines 212communicate directly or indirectly with smart-home devices 310-313, eachof which may have data-consumer (“DC”), data-source (“DS”),services-consumer (“SC”), and services-source (“SS”) characteristics. Inaddition, the processing engines may access various other types ofexternal information 316, including information obtained through theInternet, various remote information sources, and even remote sensor,audio, and video feeds and sources.

Automated Schedule Learning

FIG. 4 illustrates a general class of intelligent controllers to whichthe current application is directed. The intelligent controller 402controls a device, machine, system, or organization 404 via any ofvarious different types of output control signals and receivesinformation about the controlled entity and the environment from sensoroutput received by the intelligent controller from sensors embeddedwithin the controlled entity 404, the intelligent controller 402, or inthe environment of the intelligent controller and/or controlled entity.In FIG. 4, the intelligent controller is shown connected to thecontrolled entity 404 via a wire or fiber-based communications medium406. However, the intelligent controller may be interconnected with thecontrolled entity by alternative types of communications media andcommunications protocols, including wireless communications. In manycases, the intelligent controller and controlled entity may beimplemented and packaged together as a single system that includes boththe intelligent controller and a machine, device, system, ororganization controlled by the intelligent controller. The controlledentity may include multiple devices, machines, system, or organizationsand the intelligent controller may itself be distributed among multiplecomponents and discrete devices and systems. In addition to outputtingcontrol signals to controlled entities and receiving sensor input, theintelligent controller also provides a user interface 410-413 throughwhich a human user or remote entity, including a user-operatedprocessing device or a remote automated control system, can inputimmediate-control inputs to the intelligent controller as well as createand modify the various types of control schedules. In FIG. 4, theintelligent controller provides a graphical-display component 410 thatdisplays a control schedule 416 and includes a number of inputcomponents 411-413 that provide a user interface for input ofimmediate-control directives to the intelligent controller forcontrolling the controlled entity or entities and input ofscheduling-interface commands that control display of one or morecontrol schedules, creation of control schedules, and modification ofcontrol schedules.

To summarize, the general class of intelligent controllers to which thecurrent is directed receive sensor input, output control signals to oneor more controlled entities, and provide a user interface that allowsusers to input immediate-control command inputs to the intelligentcontroller for translation by the intelligent controller into outputcontrol signals as well as to create and modify one or more controlschedules that specify desired controlled-entity operational behaviorover one or more time periods. These basic functionalities and featuresof the general class of intelligent controllers provide a basis uponwhich automated control-schedule learning, to which the currentapplication is directed, can be implemented.

FIG. 5 illustrates additional internal features of an intelligentcontroller. An intelligent controller is generally implemented using oneor more processors 502, electronic memory 504-507, and various types ofmicrocontrollers 510-512, including a microcontroller 512 andtransceiver 514 that together implement a communications port thatallows the intelligent controller to exchange data and commands with oneor more entities controlled by the intelligent controller, with otherintelligent controllers, and with various remote computing facilities,including cloud-computing facilities through cloud-computing servers.Often, an intelligent controller includes multiple differentcommunications ports and interfaces for communicating by variousdifferent protocols through different types of communications media. Itis common for intelligent controllers, for example, to use wirelesscommunications to communicate with other wireless-enabled intelligentcontrollers within an environment and with mobile-communicationscarriers as well as any of various wired communications protocols andmedia. In certain cases, an intelligent controller may use only a singletype of communications protocol, particularly when packaged togetherwith the controlled entities as a single system. Electronic memorieswithin an intelligent controller may include both volatile andnon-volatile memories, with low-latency, high-speed volatile memoriesfacilitating execution of control routines by the one or more processorsand slower, non-volatile memories storing control routines and data thatneed to survive power-on/power-off cycles. Certain types of intelligentcontrollers may additionally include mass-storage devices.

FIG. 6 illustrates a generalized computer architecture that representsan example of the type of computing machinery that may be included in anintelligent controller, server computer, and other processor-basedintelligent devices and systems. The computing machinery includes one ormultiple central processing units (“CPUs”) 602-605, one or moreelectronic memories 608 interconnected with the CPUs by aCPU/memory-subsystem bus 610 or multiple busses, a first bridge 612 thatinterconnects the CPU/memory-subsystem bus 610 with additional busses614 and 616 and/or other types of high-speed interconnection media,including multiple, high-speed serial interconnects. These busses and/orserial interconnections, in turn, connect the CPUs and memory withspecialized processors, such as a graphics processor 618, and with oneor more additional bridges 620, which are interconnected with high-speedserial links or with multiple controllers 622-627, such as controller627, that provide access to various different types of mass-storagedevices 628, electronic displays, input devices, and other suchcomponents, subcomponents, and computational resources.

FIG. 7 illustrates features and characteristics of an intelligentcontroller of the general class of intelligent controllers to which thecurrent application is directed. An intelligent controller includescontroller logic 702 generally implemented as electronic circuitry andprocessor-based computational components controlled by computerinstructions stored in physical data-storage components, includingvarious types of electronic memory and/or mass-storage devices. Itshould be noted, at the onset, that computer instructions stored inphysical data-storage devices and executed within processors comprisethe control components of a wide variety of modern devices, machines,and systems, and are as tangible, physical, and real as any othercomponent of a device, machine, or system. Occasionally, statements areencountered that suggest that computer-instruction-implemented controllogic is “merely software” or something abstract and less tangible thanphysical machine components. Those familiar with modern science andtechnology understand that this is not the case. Computer instructionsexecuted by processors must be physical entities stored in physicaldevices. Otherwise, the processors would not be able to access andexecute the instructions. The term “software” can be applied to asymbolic representation of a program or routine, such as a printout ordisplayed list of programming-language statements, but such symbolicrepresentations of computer programs are not executed by processors.Instead, processors fetch and execute computer instructions stored inphysical states within physical data-storage devices.

The controller logic accesses and uses a variety of different types ofstored information and inputs in order to generate output controlsignals 704 that control the operational behavior of one or morecontrolled entities. The information used by the controller logic mayinclude one or more stored control schedules 706, received output fromone or more sensors 708-710, immediate control inputs received throughan immediate-control interface 712, and data, commands, and otherinformation received from remote data-processing systems, includingcloud-based data-processing systems 713. In addition to generatingcontrol output 704, the controller logic provides an interface 714 thatallows users to create and modify control schedules and may also outputdata and information to remote entities, other intelligent controllers,and to users through an information-output interface.

FIG. 8 illustrates a typical control environment within which anintelligent controller operates. As discussed above, an intelligentcontroller 802 receives control inputs from users or other entities 804and uses the control inputs, along with stored control schedules andother information, to generate output control signals 805 that controloperation of one or more controlled entities 808. Operation of thecontrolled entities may alter an environment within which sensors810-812 are embedded. The sensors return sensor output, or feedback, tothe intelligent controller 802. Based on this feedback, the intelligentcontroller modifies the output control signals in order to achieve aspecified goal or goals for controlled-system operation. In essence, anintelligent controller modifies the output control signals according totwo different feedback loops. The first, most direct feedback loopincludes output from sensors that the controller can use to determinesubsequent output control signals or control-output modification inorder to achieve the desired goal for controlled-system operation. Inmany cases, a second feedback loop involves environmental or otherfeedback 816 to users which, in turn, elicits subsequent user controland scheduling inputs to the intelligent controller 802. In other words,users can either be viewed as another type of sensor that outputsimmediate-control directives and control-schedule changes, rather thanraw sensor output, or can be viewed as a component of a higher-levelfeedback loop.

There are many different types of sensors and sensor output. In general,sensor output is directly or indirectly related to some type ofparameter, machine state, organization state, computational state, orphysical environmental parameter. FIG. 9 illustrates the generalcharacteristics of sensor output. As shown in a first plot 902 in FIG.9, a sensor may output a signal, represented by curve 904, over time,with the signal directly or indirectly related to a parameter P, plottedwith respect to the vertical axis 906. The sensor may output a signalcontinuously or at intervals, with the time of output plotted withrespect to the horizontal axis 908. In certain cases, sensor output maybe related to two or more parameters. For example, in plot 910, a sensoroutputs values directly or indirectly related to two differentparameters P1 and P2, plotted with respect to axes 912 and 914,respectively, over time, plotted with respect to vertical axis 916. Inthe following discussion, for simplicity of illustration and discussion,it is assumed that sensors produce output directly or indirectly relatedto a single parameter, as in plot 902 in FIG. 9. In the followingdiscussion, the sensor output is assumed to be a set of parameter valuesfor a parameter P. The parameter may be related to environmentalconditions, such as temperature, ambient light level, sound level, andother such characteristics. However, the parameter may also be theposition or positions of machine components, the data states ofmemory-storage address in data-storage devices, the current drawn from apower supply, the flow rate of a gas or fluid, the pressure of a gas orfluid, and many other types of parameters that comprise usefulinformation for control purposes.

FIGS. 10A-D illustrate information processed and generated by anintelligent controller during control operations. All the figures showplots, similar to plot 902 in FIG. 9, in which values of a parameter oranother set of control-related values are plotted with respect to avertical axis and time is plotted with respect to a horizontal axis.FIG. 10A shows an idealized specification for the results ofcontrolled-entity operation. The vertical axis 1002 in FIG. 10Arepresents a specified parameter value, Ps. For example, in the case ofan intelligent thermostat, the specified parameter value may betemperature. For an irrigation system, by contrast, the specifiedparameter value may be flow rate. FIG. 10A is the plot of a continuouscurve 1004 that represents desired parameter values, over time, that anintelligent controller is directed to achieve through control of one ormore devices, machines, or systems. The specification indicates that theparameter value is desired to be initially low 1006, then rise to arelatively high value 1008, then subside to an intermediate value 1010,and then again rise to a higher value 1012. A control specification canbe visually displayed to a user, as one example, as a control schedule.

FIG. 10B shows an alternate view, or an encoded-data view, of a controlschedule corresponding to the control specification illustrated in FIG.10A. The control schedule includes indications of a parameter-valueincrease 1016 corresponding to edge 1018 in FIG. 10A, a parameter-valuedecrease 1020 corresponding to edge 1022 in FIG. 10A, and aparameter-value increase 1024 corresponding to edge 1016 in FIG. 10A.The directional arrows plotted in FIG. 10B can be considered to besetpoints, or indications of desired parameter changes at particularpoints in time within some period of time.

The control schedules learned by an intelligent controller represent asignificant component of the results of automated learning. The learnedcontrol schedules may be encoded in various different ways and stored inelectronic memories or mass-storage devices within the intelligentcontroller, within the system controlled by the intelligent controller,or within remote data-storage facilities, includingcloud-computing-based data-storage facilities. In many cases, thelearned control schedules may be encoded and stored in multiplelocations, including control schedules distributed among internalintelligent-controller memory and remote data-storage facilities. Asetpoint change may be stored as a record with multiple fields,including fields that indicate whether the setpoint change is asystem-generated setpoint or a user-generated setpoint, whether thesetpoint change is an immediate-control-input setpoint change or ascheduled setpoint change, the time and date of creation of the setpointchange, the time and date of the last edit of the setpoint change, andother such fields. In addition, a setpoint may be associated with two ormore parameter values. As one example, a range setpoint may indicate arange of parameter values within which the intelligent controller shouldmaintain a controlled environment. Setpoint changes are often referredto as “setpoints.”

FIG. 10C illustrates the control output by an intelligent controllerthat might result from the control schedule illustrated in FIG. 10B. Inthis figure, the magnitude of an output control signal is plotted withrespect to the vertical axis 1026. For example, the control output maybe a voltage signal output by an intelligent thermostat to a heatingunit, with a high-voltage signal indicating that the heating unit shouldbe currently operating and a low-voltage output indicating that theheating system should not be operating. Edge 1028 in FIG. 10Ccorresponds to setpoint 1016 in FIG. 10B. The width of the positivecontrol output 1030 may be related to the length, or magnitude, of thedesired parameter-value change, indicated by the length of setpointarrow 1016. When the desired parameter value is obtained, theintelligent controller discontinues output of a high-voltage signal, asrepresented by edge 1032. Similar positive output control signals 1034and 1036 are elicited by setpoints 1020 and 1024 in FIG. 10B.

Finally, FIG. 10D illustrates the observed parameter changes, asindicated by sensor output, resulting from control, by the intelligentcontroller, of one or more controlled entities. In FIG. 10D, the sensoroutput, directly or indirectly related to the parameter P, is plottedwith respect to the vertical axis 1040. The observed parameter value isrepresented by a smooth, continuous curve 1042. Although this continuouscurve can be seen to be related to the initial specification curve,plotted in FIG. 10A, the observed curve does not exactly match thatspecification curve. First, it may take a finite period of time 1044 forthe controlled entity to achieve the parameter-valued change representedby setpoint 1016 in the control schedule plotted in FIG. 10B. Also, oncethe parameter value is obtained, and the controlled entity directed todiscontinue operation, the parameter value may begin to fall 1046,resulting in a feedback-initiated control output to resume operation ofthe controlled entity in order to maintain the desired parameter value.Thus, the desired high-level constant parameter value 1008 in FIG. 10Amay, in actuality, end up as a time-varying curve 1048 that does notexactly correspond to the control specification 1004. The first level offeedback, discussed above with reference to FIG. 8, is used by theintelligent controller to control one or more control entities so thatthe observed parameter value, over time, as illustrated in FIG. 10D,matches the specified time behavior of the parameter in FIG. 10A asclosely as possible. The second level feedback control loop, discussedabove with reference to FIG. 8, may involve alteration of thespecification, illustrated in FIG. 10A, by a user, over time, either bychanges to stored control schedules or by input of immediate-controldirectives, in order to generate a modified specification that producesa parameter-value/time curve reflective of a user's desired operationalresults.

There are many types of controlled entities and associated controllers.In certain cases, control output may include both an indication ofwhether the controlled entity should be currently operational as well asan indication of a level, throughput, or output of operation when thecontrolled entity is operational. In other cases, the control out may besimply a binary activation/deactivation signal. For simplicity ofillustration and discussion, the latter type of control output isassumed in the following discussion.

FIGS. 11A-E provide a transition-state-diagram-based illustration ofintelligent-controller operation. In these diagrams, the disk-shapedelements, or nodes, represent intelligent-controller states and thecurved arrows interconnecting the nodes represent state transitions.FIG. 11A shows one possible state-transition diagram for an intelligentcontroller. There are four main states 1102-1105. These states include:(1) a quiescent state 1102, in which feedback from sensors indicate thatno controller outputs are currently needed and in which the one or morecontrolled entities are currently inactive or in maintenance mode; (2)an awakening state 1103, in which sensor data indicates that an outputcontrol may be needed to return one or more parameters to within adesired range, but the one or more controlled entities have not yet beenactivated by output control signals; (3) an active state 1104, in whichthe sensor data continue to indicate that observed parameters areoutside desired ranges and in which the one or more controlled entitieshave been activated by control output and are operating to return theobserved parameters to the specified ranges; and (4) an incipientquiescent state 1105, in which operation of the one or more controlledentities has returned the observed parameter to specified ranges butfeedback from the sensors has not yet caused the intelligent controllerto issue output control signals to the one or more controlled entitiesto deactivate the one or more controlled entities. In general, statetransitions flow in a clockwise direction, with the intelligentcontroller normally occupying the quiescent state 1102, but periodicallyawakening, in step 1103, due to feedback indications in order toactivate the one or more controlled entities, in state 1104, to returnobserved parameters back to specified ranges. Once the observedparameters have returned to specified ranges, in step 1105, theintelligent controller issues deactivation output control signals to theone or more controlled entities, returning to the quiescent state 1102.

Each of the main-cycle states 1102-1105 is associated with twoadditional states: (1) a schedule-change state 1106-1109 and acontrol-change state 1110-1113. These states are replicated so that eachmain-cycle state is associated with its own pair of schedule-change andcontrol-change states. This is because, in general, schedule-change andcontrol-change states are transient states, from which the controllerstate returns either to the original main-cycle state from which theschedule-change or control-change state was reached by a previoustransition or to a next main-cycle state in the above-described cycle.Furthermore, the schedule-change and control-change states are a type ofparallel, asynchronously operating state associated with the main-cyclestates. A schedule-change state represents interaction between theintelligent controller and a user or other remote entity carrying outcontrol-schedule-creation, control-schedule-modification, orcontrol-schedule-management operations through a displayed-scheduleinterface. The control-change states represent interaction of a user orother remote entity to the intelligent controller in which the user orother remote entity inputs immediate-control commands to the intelligentcontroller for translation into output control signals to the one ormore controlled entities.

FIG. 11B is the same state-transition diagram shown in FIG. 11A, withthe addition of circled, alphanumeric labels, such as circled,alphanumeric label 1116, associated with each transition. FIG. 11Cprovides a key for these transition labels. FIGS. 11B-C thus togetherprovide a detailed illustration of both the states and state transitionsthat together represent intelligent-controller operation.

To illustrate the level of detail contained in FIGS. 11B-C, consider thestate transitions 1118-1120 associated with states 1102 and 1106. As canbe determined from the table provided in FIG. 11C, the transition 1118from state 1102 to state 1106 involves a control-schedule change made byeither a user, a remote entry, or by the intelligent controller itselfto one or more control schedules stored within, or accessible to, theintelligent controller. In general, following the schedule change,operation transitions back to state 1102 via transition 1119. However,in the relatively unlikely event that the schedule change has resultedin sensor data that was previously within specified ranges now fallingoutside newly specified ranges, the state transitions instead, viatransition 1120, to the awakening state 1103.

Automated control-schedule learning by the intelligent controller, infact, occurs largely as a result of intelligent-controller operationwithin the schedule-change and control-change states. Immediate-controlinputs from users and other remote entities, resulting in transitions tothe control-change states 1110-1113, provide information from which theintelligent controller learns, over time, how to control the one or morecontrolled entities in order to satisfy the desires and expectations ofone or more users or remote entities. The learning process is encoded,by the intelligent controller, in control-schedule changes made by theintelligent controller while operating in the schedule-change states1106-1109. These changes are based on recorded immediate-control inputs,recorded control-schedule changes, and current and historicalcontrol-schedule information. Additional sources of information forlearning may include recorded output control signals and sensor inputsas well as various types of information gleaned from external sources,including sources accessible through the Internet. In addition to thepreviously described states, there is also an initial state or states1130 that represent a first-power-on state or state following a reset ofthe intelligent controller. Generally, a boot operation followed by aninitial-configuration operation or operations leads from the one or moreinitial states 1130, via transitions 1132 and 1134, to one of either thequiescent state 1102 or the awakening state 1103.

FIGS. 11D-E illustrate, using additional shading of the states in thestate-transition diagram shown in FIG. 11A, two modes of automatedcontrol-schedule learning carried out by an intelligent controller towhich the current application is directed. The first mode, illustratedin FIG. 11D, is a steady-state mode. The steady-state mode seeks optimalor near-optimal control with minimal immediate-control input. Whilelearning continues in the steady-state mode, the learning is implementedto respond relatively slowly and conservatively to immediate-controlinput, sensor input, and input from external information sources withthe presumption that steady-state learning is primarily tailored tosmall-grain refinement of control operation and tracking of relativelyslow changes in desired control regimes over time. In steady-statelearning and general intelligent-controller operation, the mostdesirable state is the quiescent state 1102, shown crosshatched in FIG.11D to indicate this state as the goal, or most desired state, ofsteady-state operation. Light shading is used to indicate that the othermain-cycle states 1103-1105 have neutral or slighted favored status inthe steady-state mode of operation. Clearly, these states are needed forintermediate or continuous operation of controlled entities in order tomaintain one or more parameters within specified ranges, and to trackscheduled changes in those specified ranges. However, these states areslightly disfavored in that, in general, a minimal number, or minimalcumulative duration, of activation and deactivation cycles of the one ormore controlled entities often leads to most optimal control regimes,and minimizing the cumulative time of activation of the one or morecontrolled entities often leads to optimizing the control regime withrespect to energy and/or resource usage. In the steady-state mode ofoperation, the schedule-change and control-change states 1110-1113 arehighly disfavored, because the intent of automated control-schedulelearning is for the intelligent controller to, over time, devise one ormore control schedules that accurately reflect a user's or other remoteentity's desired operational behavior. While, at times, these states maybe temporarily frequently inhabited as a result of changes in desiredoperational behavior, changes in environmental conditions, or changes inthe controlled entities, a general goal of automated control-schedulelearning is to minimize the frequency of both schedule changes andimmediate-control inputs. Minimizing the frequency of immediate-controlinputs is particularly desirable in many optimization schemes.

FIG. 11E, in contrast to FIG. 11D, illustrates an aggressive-learningmode in which the intelligent controller generally operates for a shortperiod of time following transitions within the one or more initialstates 730 to the main-cycle states 1102-1103. During theaggressive-learning mode, in contrast to steady-state operational modeshown in FIG. 11D, the quiescent state 1102 is least favored and theschedule-change and control-change states 1106-1113 are most favored,with states 1103-1105 having neutral desirability. In theaggressive-learning mode or phase of operation, the intelligentcontroller seeks frequent immediate-control inputs and schedule changesin order to quickly and aggressively acquire one or more initial controlschedules. As discussed below, by using relatively rapidimmediate-control-input relaxation strategies, the intelligentcontroller, while operating in aggressive-learning mode, seeks to compela user or other remote entity to provide immediate-control inputs atrelatively short intervals in order to quickly determine the overallshape and contour of an initial control schedule. Following completionof the initial aggressive learning and generation of adequate initialcontrol schedules, relative desirability of the various states revertsto those illustrated in FIG. 11D as the intelligent controller begins torefine control schedules and track longer-term changes in controlspecifications, the environment, the control system, and other suchfactors. Thus, the automated control-schedule-learning methods andintelligent controllers incorporating these methods to which the currentapplication is directed feature an initial aggressive-learning mode thatis followed, after a relatively short period of time, by a long-term,steady-state learning mode.

FIG. 12 provide a state-transition diagram that illustrates automatedcontrol-schedule learning. Automated learning occurs during normalcontroller operation, illustrated in FIGS. 11A-C, and thus thestate-transition diagram shown in FIG. 12 describes operation behaviorsof an intelligent controller that occur in parallel with theintelligent-controller operation described in FIGS. 11A-C. Following oneor more initial states 1202, corresponding to the initial states 1130 inFIG. 11B, the intelligent controller enters an initial-configurationlearning state 1204 in which the intelligent controller attempts tocreate one or more initial control schedules based on one or more ofdefault control schedules stored within the intelligent controller oraccessible to the intelligent controller, an initial-schedule-creationdialog with a user or other remote entity through a schedule-creationinterface, by a combination of these two approaches, or by additionalapproaches. The initial-configuration learning mode 1204 occurs inparallel with transitions 1132 and 1134 in FIG. 11B. During theinitial-learning mode, learning from manually entered setpoint changesdoes not occur, as it has been found that users often make many suchchanges inadvertently, as they manipulate interface features to explorethe controller's features and functionalities.

Following initial configuration, the intelligent controller transitionsnext to the aggressive-learning mode 1206, discussed above withreference to FIG. 11E. The aggressive-learning mode 1206 is alearning-mode state which encompasses most or all states except forstate 1130 of the states in FIG. 11B. In other words, theaggressive-learning-mode state 1206 is a learning-mode state parallel tothe general operational states discussed in FIGS. 11A-E. As discussedabove, during aggressive learning, the intelligent controller attemptsto create one or more control schedules that are at least minimallyadequate to specify operational behavior of the intelligent controllerand the entities which it controls based on frequent input from users orother remote entities. Once aggressive learning is completed, theintelligent controller transitions forward through a number ofsteady-state learning phases 1208-1210. Each transition downward, in thestate-transition diagram shown in FIG. 12, through the series ofsteady-state learning-phase states 1208-1210, is accompanied by changesin learning-mode parameters that result in generally slower, moreconservative approaches to automated control-schedule learning as theone or more control schedules developed by the intelligent controller inprevious learning states become increasingly accurate and reflective ofuser desires and specifications. The determination of whether or notaggressive learning is completed may be made based on a period of time,a number of information-processing cycles carried out by the intelligentcontroller, by determining whether the complexity of the current controlschedule or schedules is sufficient to provide a basis for slower,steady-state learning, and/or on other considerations, rules, andthresholds. It should be noted that, in certain implementations, theremay be multiple aggressive-learning states.

FIG. 13 illustrates time frames associated with an example controlschedule that includes shorter-time-frame sub-schedules. The controlschedule is graphically represented as a plot with the horizontal axis1302 representing time. The vertical axis 1303 generally represents oneor more parameter values. As discussed further, below, a controlschedule specifies desired parameter values as a function of time. Thecontrol schedule may be a discrete set of values or a continuous curve.The specified parameter values are either directly or indirectly relatedto observable characteristics in environment, system, device, machine,or organization that can be measured by, or inferred from measurementsobtained from, any of various types of sensors. In general, sensoroutput serves as at least one level of feedback control by which anintelligent controller adjusts the operational behavior of a device,machine, system, or organization in order to bring observed parametervalues in line with the parameter values specified in a controlschedule. The control schedule used as an example in the followingdiscussion is incremented in hours, along the horizontal axis, andcovers a time span of one week. The control schedule includes sevensub-schedules 1304-1310 that correspond to days. As discussed furtherbelow, in an example intelligent controller, automated control-schedulelearning takes place at daily intervals, with a goal of producing arobust weekly control schedule that can be applied cyclically, weekafter week, over relatively long periods of time. As also discussedbelow, an intelligent controller may learn even longer-period controlschedules, such as yearly control schedules, with monthly, weekly,daily, and even hourly sub-schedules organized hierarchically below theyearly control schedule. In certain cases, an intelligent controller maygenerate and maintain shorter-time-frame control schedules, includinghourly control schedules, minute-based control schedules, or evencontrol schedules incremented in milliseconds and microseconds. Controlschedules are, like the stored computer instructions that togethercompose control routines, tangible, physical components of controlsystems. Control schedules are stored as physical states in physicalstorage media. Like control routines and programs, control schedules arenecessarily tangible, physical control-system components that can beaccessed and used by processor-based control logic and control systems.

FIGS. 14A-C show three different types of control schedules. In FIG.14A, the control schedule is a continuous curve 1402 representing aparameter value, plotted with respect to the vertical axis 1404, as afunction of time, plotted with respect to the horizontal axis 1406. Thecontinuous curve comprises only horizontal and vertical sections.Horizontal sections represent periods of time at which the parameter isdesired to remain constant and vertical sections represent desiredchanges in the parameter value at particular points in time. This is asimple type of control schedule and is used, below, in various examplesof automated control-schedule learning. However, automatedcontrol-schedule-learning methods can also learn more complex types ofschedules. For example, FIG. 14B shows a control schedule that includesnot only horizontal and vertical segments, but arbitrarily angledstraight-line segments. Thus, a change in the parameter value may bespecified, by such a control schedule, to occur at a given rate, ratherthan specified to occur instantaneously, as in the simple controlschedule shown in FIG. 14A. Automated-control-schedule-learning methodsmay also accommodate smooth-continuous-curve-based control schedules,such as that shown in FIG. 14C. In general, the characterization anddata encoding of smooth, continuous-curve-based control schedules, suchas that shown in FIG. 14C, is more complex and includes a greater amountof stored data than the simpler control schedules shown in FIGS. 14B and14A.

In the following discussion, it is generally assumed that a parametervalue tends to relax towards lower values in the absence of systemoperation, such as when the parameter value is temperature and thecontrolled system is a heating unit. However, in other cases, theparameter value may relax toward higher values in the absence of systemoperation, such as when the parameter value is temperature and thecontrolled system is an air conditioner. The direction of relaxationoften corresponds to the direction of lower resource or expenditure bythe system. In still other cases, the direction of relaxation may dependon the environment or other external conditions, such as when theparameter value is temperature and the controlled system is an HVACsystem including both heating and cooling functionality.

Turning to the control schedule shown in FIG. 14A, thecontinuous-curve-represented control schedule 1402 may be alternativelyencoded as discrete setpoints corresponding to vertical segments, oredges, in the continuous curve. A continuous-curve control schedule isgenerally used, in the following discussion, to represent a storedcontrol schedule either created by a user or remote entity via aschedule-creation interface provided by the intelligent controller orcreated by the intelligent controller based on already-existing controlschedules, recorded immediate-control inputs, and/or recorded sensordata, or a combination of these types of information.

Immediate-control inputs are also graphically represented inparameter-value versus time plots. FIGS. 15A-G show representations ofimmediate-control inputs that may be received and executed by anintelligent controller, and then recorded and overlaid onto controlschedules, such as those discussed above with reference to FIGS. 14A-C,as part of automated control-schedule learning. An immediate-controlinput is represented graphically by a vertical line segment that ends ina small filled or shaded disk. FIG. 15A shows representations of twoimmediate-control inputs 1502 and 1504. An immediate-control input isessentially equivalent to an edge in a control schedule, such as thatshown in FIG. 14A, that is input to an intelligent controller by a useror remote entity with the expectation that the input control will beimmediately carried out by the intelligent controller, overriding anycurrent control schedule specifying intelligent-controller operation. Animmediate-control input is therefore a real-time setpoint input througha control-input interface to the intelligent controller.

Because an immediate-control input alters the current control schedule,an immediate-control input is generally associated with a subsequent,temporary control schedule, shown in FIG. 15A as dashed horizontal andvertical lines that form a temporary-control-schedule parameter vs. timecurve extending forward in time from the immediate-control input.Temporary control schedules 1506 and 1508 are associated withimmediate-control inputs 1502 and 1504, respectively, in FIG. 15A.

FIG. 15B illustrates an example of immediate-control input andassociated temporary control schedule. The immediate-control input 1510is essentially an input setpoint that overrides the current controlschedule and directs the intelligent controller to control one or morecontrolled entities in order to achieve a parameter value equal to thevertical coordinate of the filled disk 1512 in the representation of theimmediate-control input. Following the immediate-control input, atemporary constant-temperature control-schedule interval 1514 extendsfor a period of time following the immediate-control input, and theimmediate-control input is then relaxed by a subsequentimmediate-control-input endpoint, or subsequent setpoint 1516. Thelength of time for which the immediate-control input is maintained, ininterval 1514, is a parameter of automated control-schedule learning.The direction and magnitude of the subsequent immediate-control-inputendpoint setpoint 1516 represents one or more additionalautomated-control-schedule-learning parameters. Please note that anautomated-control-schedule-learning parameter is an adjustable parameterthat controls operation of automated control-schedule learning, and isdifferent from the one or more parameter values plotted with respect totime that comprise control schedules. The parameter values plotted withrespect to the vertical axis in the example control schedules to whichthe current discussion refers are related directly or indirectly toobservables, including environmental conditions, machines states, andthe like.

FIG. 15C shows an existing control schedule on which animmediate-control input is superimposed. The existing control schedulecalled for an increase in the parameter value P, represented by edge1520, at 7:00 a.m. (1522 in FIG. 15C). The immediate-control input 1524specifies an earlier parameter-value change of somewhat less magnitude.FIGS. 15D-G illustrate various subsequent temporary control schedulesthat may obtain, depending on various different implementations ofintelligent-controller logic and/or current values ofautomated-control-schedule-learning parameter values. In FIGS. 15D-G,the temporary control schedule associated with an immediate-controlinput is shown with dashed line segments and that portion of theexisting control schedule overridden by the immediate-control input isshown by dotted line segments. In one approach, shown in FIG. 15D, thedesired parameter value indicated by the immediate-control input 1524 ismaintained for a fixed period of time 1526 after which the temporarycontrol schedule relaxes, as represented by edge 1528, to the parametervalue that was specified by the control schedule at the point in timethat the immediate-control input is carried out. This parameter value ismaintained 1530 until the next scheduled setpoint, which corresponds toedge 1532 in FIG. 15C, at which point the intelligent controller resumescontrol according to the control schedule.

In an alternative approach shown in FIG. 15E, the parameter valuespecified by the immediate-control input 1524 is maintained 1532 until anext scheduled setpoint is reached, in this case the setpointcorresponding to edge 1520 in the control schedule shown in FIG. 15C. Atthe next setpoint, the intelligent controller resumes control accordingto the existing control schedule. This approach is often desirable,because users often expect a manually entered setpoint to remain inforce until a next scheduled setpoint change.

In a different approach, shown in FIG. 15F, the parameter valuespecified by the immediate-control input 1524 is maintained by theintelligent controller for a fixed period of time 1534, following whichthe parameter value that would have been specified by the existingcontrol schedule at that point in time is resumed 1536.

In the approach shown in FIG. 15G, the parameter value specified by theimmediate-control input 1524 is maintained 1538 until a setpoint withopposite direction from the immediate-control input is reached, at whichthe existing control schedule is resumed 1540. In still alternativeapproaches, the immediate-control input may be relaxed further, to alowest-reasonable level, in order to attempt to optimize systemoperation with respect to resource and/or energy expenditure. In theseapproaches, generally used during aggressive learning, a user iscompelled to positively select parameter values greater than, or lessthan, a parameter value associated with a minimal or low rate of energyor resource usage.

In one example implementation of automated control-schedule learning, anintelligent controller monitors immediate-control inputs and schedulechanges over the course of a monitoring period, generally coincidingwith the time span of a control schedule or sub-schedule, whilecontrolling one or more entities according to an existing controlschedule except as overridden by immediate-control inputs and inputschedule changes. At the end of the monitoring period, the recorded datais superimposed over the existing control schedule and a new provisionalschedule is generated by combining features of the existing controlschedule and schedule changes and immediate-control inputs. Followingvarious types of resolution, the new provisional schedule is promoted tothe existing control schedule for future time intervals for which theexisting control schedule is intended to control system operation.

FIGS. 16A-E illustrate one aspect of the method by which a new controlschedule is synthesized from an existing control schedule and recordedschedule changes and immediate-control inputs. FIG. 16A shows theexisting control schedule for a monitoring period. FIG. 16B shows anumber of recorded immediate-control inputs superimposed over thecontrol schedule following the monitoring period. As illustrated in FIG.16B, there are six immediate-control inputs 1602-1607. In a clusteringtechnique, clusters of existing-control-schedule setpoints andimmediate-control inputs are detected. One approach to cluster detectionis to determine all time intervals greater than a threshold lengthduring which neither existing-control-schedule setpoints norimmediate-control inputs are present, as shown in FIG. 16C. Thehorizontal, double-headed arrows below the plot, such as double-headedarrow 1610, represent the intervals of greater than the threshold lengthduring which neither existing-control-schedule setpoints norimmediate-control inputs are present in the superposition of theimmediate-control inputs onto the existing control schedule. Thoseportions of the time axis not overlapping by these intervals are thenconsidered to be clusters of existing-control-schedule setpoints andimmediate-control inputs, as shown in FIG. 16D. A first cluster 1612encompasses existing-control-schedule setpoints 1614-1616 andimmediate-control inputs 1602 and 1603. A second cluster 1620encompasses immediate-control inputs 1604 and 1605. A third cluster 1622encompasses only existing-control-schedule setpoint 1624. A fourthcluster 1626 encompasses immediate-control inputs 1606 and 1607 as wellas the existing-control-schedule setpoint 1628. In onecluster-processing method, each cluster is reduced to zero, one, or twosetpoints in a new provisional schedule generated from the recordedimmediate-control inputs and existing control schedule. FIG. 16E showsan exemplary new provisional schedule 1630 obtained by resolution of thefour clusters identified in FIG. 16D.

Cluster processing is intended to simplify the new provisional scheduleby coalescing the various existing-control-schedule setpoints andimmediate-control inputs within a cluster to zero, one, or twonew-control-schedule setpoints that reflect an apparent intent on behalfof a user or remote entity with respect to the existing control scheduleand the immediate-control inputs. It would be possible, by contrast, togenerate the new provisional schedule as the sum of theexisting-control-schedule setpoints and immediate-control inputs.However, that approach would often lead to a ragged, highly variable,and fine-grained control schedule that generally does not reflect theultimate desires of users or other remote entities and which oftenconstitutes a parameter-value vs. time curve that cannot be achieved byintelligent control. As one example, in an intelligent thermostat, twosetpoints 15 minutes apart specifying temperatures that differ by tendegrees may not be achievable by an HVAC system controlled by anintelligent controller. It may be the case, for example, that undercertain environmental conditions, the HVAC system is capable of raisingthe internal temperature of a residence by a maximum of only fivedegrees per hour. Furthermore, simple control schedules can lead to amore diverse set of optimization strategies that can be employed by anintelligent controller to control one or more entities to produceparameter values, or P values, over time, consistent with the controlschedule. An intelligent controller can then optimize the control inview of further constraints, such as minimizing energy usage or resourceutilization.

There are many possible approaches to resolving a cluster ofexisting-control-schedule setpoints and immediate-control inputs intoone or two new provisional schedule setpoints. FIGS. 17A-E illustrateone approach to resolving schedule clusters. In each of FIGS. 17A-E,three plots are shown. The first plot shows recorded immediate-controlinputs superimposed over an existing control schedule. The second plotreduces the different types of setpoints to a single generic type ofequivalent setpoints, and the final plot shows resolution of thesetpoints into zero, one, or two new provisional schedule setpoints.

FIG. 17A shows a cluster 1702 that exhibits an obvious increasingP-value trend, as can be seen when the existing-control-schedulesetpoints and immediate-control inputs are plotted together as a singletype of setpoint, or event, with directional and magnitude indicationswith respect to actual control produced from theexisting-control-schedule setpoints and immediate-control inputs 704within an intelligent controller. In this case, four out of the sixsetpoints 706-709 resulted in an increase in specified P value, withonly a single setpoint 710 resulting in a slight decrease in P value andone setpoint 712 produced no change in P value. In this and similarcases, all of the setpoints are replaced by a single setpoint specifyingan increase in P value, which can be legitimately inferred as the intentexpressed both in the existing control schedule and in theimmediate-control inputs. In this case, the single setpoint 716 thatreplaces the cluster of setpoints 704 is placed at the time of the firstsetpoint in the cluster and specifies a new P value equal to the highestP value specified by any setpoint in the cluster.

The cluster illustrated in FIG. 17B contains five setpoints 718-722. Twoof these setpoints specify a decrease in P value, two specify anincrease in P value, and one had no effect. As a result, there is noclear P-value-change intent demonstrated by the collection of setpoints,and therefore the new provisional schedule 724 contains no setpointsover the cluster interval, with the P value maintained at the initial Pvalue of the existing control schedule within the cluster interval.

FIG. 17C shows a cluster exhibiting a clear downward trend, analogous tothe upward trend exhibited by the clustered setpoints shown in FIG. 17A.In this case, the four cluster setpoints are replaced by a single newprovisional schedule setpoint 726 at a point in time corresponding tothe first setpoint in the cluster and specifying a decrease in P valueequivalent to the lowest P value specified by any of the setpoints inthe cluster.

In FIG. 17D, the cluster includes three setpoints 730-732. The setpointcorresponding to the existing-control-schedule setpoint 730 and asubsequent immediate-control setpoint 731 indicate a clear intent toraise the P value at the beginning of the cluster interval and the finalsetpoint 732 indicates a clear intent to lower the P value at the end ofthe cluster interval. In this case, the three setpoints are replaced bytwo setpoints 734 and 736 in the new provisional schedule that mirrorthe intent inferred from the three setpoints in the cluster. FIG. 17Eshows a similar situation in which three setpoints in the cluster arereplaced by two new-provisional-schedule setpoints 738 and 740, in thiscase representing a temporary lowering and then subsequent raising ofthe P value as opposed to the temporary raising and subsequent loweringof the P value in the new provisional schedule in FIG. 17B.

There are many different computational methods that can recognize thetrends of clustered setpoints discussed with reference to FIGS. 17A-E.These trends provide an example of various types of trends that may becomputationally recognized. Different methods and strategies for clusterresolution are possible, including averaging, curve fitting, and othertechniques. In all cases, the goal of cluster resolution is to resolvemultiple setpoints into a simplest possible set of setpoints thatreflect a user's intent, as judged from the existing control scheduleand the immediate-control inputs.

FIGS. 18A-B illustrate the effect of a prospective schedule changeentered by a user during a monitoring period. In FIGS. 18A-B, and insubsequent figures, a schedule-change input by a user is represented bya vertical line 1802 ending in a small filled disk 1804 indicating aspecified P value. The setpoint is placed with respect to the horizontalaxis at a time at which the setpoint is scheduled to be carried out. Ashort vertical line segment 1806 represents the point in time that theschedule change was made by a user or remote entity, and a horizontalline segment 1808 connects the time of entry with the time for executionof the setpoint represented by vertical line segments 1806 and 1802,respectively. In the case shown in FIG. 18A, a user altered the existingcontrol schedule, at 7:00 a.m. 1810, to include setpoint 1802 at 11:00a.m. In cases such as those shown in FIG. 18A, where the schedule changeis prospective and where the intelligent controller can control one ormore entities according to the changed control schedule within the samemonitoring period, the intelligent controller simply changes the controlschedule, as indicated in FIG. 18B, to reflect the schedule change. Inone automated-control-schedule-learning method, therefore, prospectiveschedule changes are not recorded. Instead, the existing controlschedule is altered to reflect a user's or remote entity's desiredschedule change.

FIGS. 19A-B illustrate the effect of a retrospective schedule changeentered by a user during a monitoring period. In the case shown in FIG.19A, a user input three changes to the existing control schedule at 6:00p.m. 1902, including deleting an existing setpoint 1904 and adding twonew setpoints 1906 and 1908. All of these schedule changes would impactonly a future monitoring period controlled by the modified controlschedule, since the time at which they were entered is later than thetime at which the changes in P value are scheduled to occur. For thesetypes of schedule changes, the intelligent controller records theschedule changes in a fashion similar to the recording ofimmediate-control inputs, including indications of the fact that thistype of setpoint represents a schedule change made by a user through aschedule-modification interface rather than an immediate-control input.

FIG. 19B shows a new provisional schedule that incorporates the schedulechanges shown in FIG. 19A. In general, schedule changes are givenrelatively large deference by the currently describedautomated-control-schedule-learning method. Because a user has taken thetime and trouble to make schedule changes through a schedule-changeinterface, it is assumed that the schedule changes are stronglyreflective of the user's desires and intentions. As a result, as shownin FIG. 19B, the deletion of existing setpoint 1904 and the addition ofthe two new setpoints 1906 and 1908 are entered into the existingcontrol schedule to produce the new provisional schedule 1910. Edge 1912corresponds to the schedule change represented by setpoint 1906 in FIG.19A and edge 1914 corresponds to the schedule change represented bysetpoint 1908 in FIG. 19A. In summary, either for prospective schedulechanges or retrospective schedule changes made during a monitoringperiod, the schedule changes are given great deference duringlearning-based preparation of a new provisional schedule thatincorporates both the existing control schedule and recordedimmediate-control inputs and schedule changes made during the monitoringperiod.

FIGS. 20A-C illustrate overlay of recorded data onto an existing controlschedule, following completion of a monitoring period, followed byclustering and resolution of clusters. As shown in FIG. 20A, a user hasinput six immediate-control inputs 2004-2009 and two retrospectiveschedule changes 2010 and 2012 during the monitoring period, which areoverlain or superimposed on the existing control schedule 2002. As shownin FIG. 20B, clustering produces four clusters 2014-2017. FIG. 20C showsthe new provisional schedule obtained by resolution of the clusters.Cluster 2014, with three existing-control-schedule setpoints and twoimmediate-control setpoints, is resolved to new-provisional-schedulesetpoints 2020 and 2022. Cluster 2 (2015 in FIG. 20B), containing twoimmediate-control setpoints and two retrospective-schedule setpoints, isresolved to setpoints 2024 and 2026. Cluster 3 (2016 in FIG. 20B) isresolved to the existing-control-schedule setpoint 2028, and cluster 4(2017 in FIG. 20B), containing two immediate-control setpoints and anexisting-control-schedule setpoint, is resolved to setpoint 2030. Inpreparation for a subsequent schedule-propagation step, each of thenew-provisional-schedule setpoints is labeled with an indication ofwhether or not the setpoint parameter value is derived from animmediate-control setpoint or from either an existing-control-schedulesetpoint or retrospective schedule-change setpoint. The latter twocategories are considered identical, and setpoints of those categoriesare labeled with the character “s” in FIG. 20C, while the setpoints withtemperatures derived from immediate-control setpoints, 2020 and 2022,are labeled “i.” As discussed further, below, only setpoints labeled “i”are propagated to additional, related sub-schedules of a higher-levelcontrol schedule.

An additional step that may follow clustering and cluster resolution andprecede new-provisional-schedule propagation, in certainimplementations, involves spreading apart setpoints derived fromimmediate-control setpoints in the new provisional schedule. FIGS. 21A-Billustrate the setpoint-spreading operation. FIG. 21A shows a newprovisional schedule with setpoints labeled, as discussed above withreference to FIG. 20C, with either “s” or “i” in order to indicate theclass of setpoints from which the setpoints were derived. In this newprovisional schedule 2102, two setpoints labeled “i” 2104 and 2106 areseparated by a time interval 2108 of length less than a threshold timeinterval for separation purposes. The spreading operation detects pairsof “i” labeled setpoints that are separated, in time, by less than thethreshold time interval and moves the latter setpoint of the pairforward, in time, so that the pair of setpoints are separated by atleast a predetermined fixed-length time interval 2110 in FIG. 21B. In aslightly more complex spreading operation, in the case that the lattersetpoint of the pair would be moved closer than the threshold time to asubsequent setpoint, the latter setpoint may be moved to a point in timehalfway between the first setpoint of the pair and the subsequentsetpoint. The intent of the spreading operation is to ensure adequateseparation between setpoints for schedule simplicity and in order toproduce a control schedule that can be realized underintelligent-controller control of a system.

A next operation carried out by the currently discussedautomated-control-schedule-learning method is propagation of a newprovisional sub-schedule, created, as discussed above, following amonitoring period, to related sub-schedules in a higher-level controlschedule. Schedule propagation is illustrated in FIGS. 22A-B. FIG. 22Ashows a higher-level control schedule 2202 that spans a week in time andthat includes daily sub-schedules, such as the Saturday sub-schedule2204. In FIG. 22A, the Monday sub-schedule 2206 has recently beenreplaced by a new provisional Monday sub-schedule following the end of amonitoring period, indicated in FIG. 22A by crosshatching oppositelyslanted from the crosshatching of the sub-schedules corresponding toother days of the week. As shown in FIG. 22B, the schedule-propagationtechnique used in the currently discussedautomated-control-schedule-learning method involves propagating the newprovisional Monday sub-schedule 2206 to other, related sub-schedules2208-2211 in the higher-level control schedule 2202. In this case,weekday sub-schedules are considered to be related to one another, asare weekend sub-schedules, but weekend sub-schedules are not consideredto be related to weekday sub-schedules. Sub-schedule propagationinvolves overlaying the “i”-labeled setpoints in the new provisionalschedule 2206 over related existing control schedules, in this casesub-schedules 2208-2211, and then resolving the setpoint-overlaidexisting control schedules to produce new provisional sub-schedules forthe related sub-schedules. In FIG. 22B, overlaying of “i”-labeledsetpoints from new provisional sub-schedule 2206 onto the relatedsub-schedules 2208-2211 is indicated by bi-directional crosshatching.Following resolution of these overlaid setpoints and existingsub-schedules, the entire higher-level control schedule 2202 is thenconsidered to be the current existing control schedule for theintelligent controller. In other words, following resolution, the newprovisional sub-schedules are promoted to existing sub-schedules. Incertain cases, the sub-schedule propagation rules may change, over time.As one example, propagation may occur to all days, initially, of aweekly schedule but may then more selectively propagate weekdaysub-schedules to weekdays and week-end-day sub-schedules toweek-end-days. Other such rules may be employed for propagation ofsub-schedules.

As discussed above, there can be multiple hierarchical layers of controlschedules and sub-schedules maintained by an intelligent controller, aswell as multiple sets of hierarchically related control schedules. Inthese cases, schedule propagation may involve relatively more complexpropagation rules for determining to which sub-schedules a newly createdprovisional sub-schedule should be propagated. Although propagation isshown, in FIG. 22B, in the forward direction in time, propagation of anew provisional schedule or new provisional sub-schedule may be carriedout in either a forward or reverse direction with respect to time. Ingeneral, new-provisional-schedule propagation is governed by rules or bytables listing those control schedules and sub-schedules considered tobe related to each control schedule and/or sub-schedule.

FIGS. 23A-C illustrate new-provisional-schedule propagation usingP-value vs. t control-schedule plots. FIG. 23A shows an existing controlschedule 2302 to which the “i”-labeled setpoints in a new provisionalschedule are propagated. FIG. 23B shows the propagated setpoints with“i” labels overlaid onto the control schedule shown in FIG. 23A. Twosetpoints 2304 and 2306 are overlaid onto the existing control schedule2302. The existing control schedule includes four existing setpoints2308-2311. The second of the propagated setpoints 2306 lowers theparameter value to a level 2312 greater than the correspondingparameter-value level 2314 of the existing control schedule 2302, andtherefore overrides the existing control schedule up to existingsetpoint 2310. In this simple case, no further adjustments are made, andthe propagated setpoints are incorporated in the existing controlschedule to produce a new provisional schedule 2316 shown in FIG. 23C.When setpoints have been propagated to all related control schedules orsub-schedules, and new provisional schedules and sub-schedules aregenerated for them, the propagation step terminates, and all of the newprovisional schedules and sub-schedules are together considered to be anew existing higher-level control schedule for the intelligentcontroller.

Following propagation and overlaying of “i”-labeled setpoints onto a newprovisional schedule to a related sub-schedule or control schedule, asshown in FIG. 23B, numerous rules may be applied to the overlyingsetpoints and existing control schedule in order to simplify and to makerealizable the new provisional schedule generated from the propagatedsetpoints and existing control schedule. FIGS. 24A-I illustrate a numberof example rules used to simplify a existing control schedule overlaidwith propagated setpoints as part of the process of generating a newprovisional schedule. Each of FIGS. 24A-I include two P-value vs. tplots, the first showing a propagated setpoint overlying a existingcontrol schedule and the second showing resolution of the propagatedsetpoint to generate a portion of a new provisional schedule obtained byresolving a propagated setpoint.

The first, left-hand P-value vs. t plot 2402 in FIG. 24A shows apropagated setpoint 2404 overlying an existing control schedule 2405.FIG. 24A also illustrates terminology used in describing many of theexample rules used to resolve propagated setpoints with existing controlschedules. In FIG. 24A a first existing setpoint, pe1 2406, precedes thepropagated setpoint 2404 in time by a length of time a 2407 and a secondexisting setpoint of the existing control schedule, pe2 2408, followsthe propagated setpoint 2404 in time by a length of time b 2409. TheP-value difference between the first existing-control-schedule setpoint2406 and the propagated setpoint 2404 is referred to as “ΔP” 2410. Theright-hand P-value vs. t plot 2412 shown in FIG. 24A illustrates a firstpropagated-setpoint-resolution rule. As shown in this Figure, when ΔP isless than a threshold ΔP and b is less than a threshold Δt, then thepropagated setpoint is deleted. Thus, resolution of the propagatedsetpoint with the existing control schedule, by rule 1, removes thepropagated setpoint, as shown in the right-hand side of FIG. 24A.

FIGS. 24B-I illustrate an additional example ofpropagated-setpoint-resolution rules in similar fashion to theillustration of the first propagated-setpoint-resolution rule in FIG.24A. FIG. 24B illustrates a rule that, when b is less than a thresholdΔt and when the first rule illustrated in FIG. 24A does not apply, thenthe new propagated setpoint 2414 is moved ahead in time by a value Δt22416 from existing setpoint pe1 and existing setpoint pe2 is deleted.

FIG. 24C illustrates a third rule applied when neither of the first tworules are applicable to a propagated setpoint. If a is less than athreshold value Δt, then the propagated setpoint is moved back in timeby a predetermined value Δt3 from pe2 and the existing setpoint pe1 isdeleted.

FIG. 24D illustrates a fourth row applicable when none of the firstthree rules can be applied to a propagated setpoint. In this case, the Pvalue of the propagated setpoint becomes the P value for the existingsetpoint pe1 and the propagated setpoint is deleted.

When none of the first four rules, described above with reference toFIGS. 24A-D, are applicable, then additional rules may be tried in orderto resolve a propagated setpoint with an existing control schedule. FIG.24E illustrates a fifth rule. When b is less than a threshold Δt and ΔPis less than a threshold Δp, then, as shown in FIG. 24E, the propagatedsetpoint 2424 is deleted. In other words, a propagated setpoint tooclose to an existing control schedule setpoint is not incorporated intothe new provisional control schedule. The existing setpoints may also bereconsidered, during propagated-setpoint resolution. For example, asshown in FIG. 24F, when a second existing setpoint pe2 that occurs aftera first existing setpoint pe1 results in a change in the parameter valueΔP less than a threshold ΔP, then the second existing setpoint pe2 maybe removed. Such proximal existing setpoints may arise due to thedeference given to schedule changes following previous monitoringperiods. Similarly, as shown in FIG. 24G, when a propagated setpointfollows an existing setpoint, and the change in the parameter value ΔPproduced by the propagated setpoint is less than a threshold ΔP value,then the propagated setpoint is deleted. As shown in FIG. 24H, twoexisting setpoints that are separated by less than a threshold Δt valuemay be resolved into a single setpoint coincident with the first of thetwo existing setpoints. Finally, in similar fashion, a propagatedsetpoint that is too close, in time, to an existing setpoint may bedeleted.

In certain implementations, a significant distinction is made betweenuser-entered setpoint changes and automatically generated setpointchanges. The former setpoint changes are referred to as “anchorsetpoints,” and are not overridden by learning. In many cases, usersexpect that the setpoints which they manually enter should not bechanged. Additional rules, heuristics, and consideration can be used todifferentiate setpoint changes for various levels of automatedadjustment during both aggressive and steady-state learning. It shouldalso be noted that setpoints associated with two parameter values thatindicate a parameter-value range may be treated in different ways duringcomparison operations used in pattern matching and other automatedlearning calculations and determinations. For example, a range setpointchange may need to match another range setpoint change in bothparameters to be deemed to be equivalent or identical.

Next, an example implementation of an intelligent controller thatincorporates the above-described automated-control-schedule-learningmethod is provided. FIGS. 25A-M illustrate an example implementation ofan intelligent controller that incorporates the above-describedautomated-control-schedule-learning method. At the onset, it should benoted that the following implementation is but one of many differentpossible implementations that can be obtained by varying any of manydifferent design and implementation parameters, including modularorganization, control structures, data structures, programming language,hardware components, firmware, and other such design and implementationparameters. Many different types of control schedules may be used bydifferent types of intelligent controllers applied to different controldomains. Automated-control-schedule-learning methods incorporated intointelligent-controller logic may significantly vary depending on thetypes and numbers of control schedules that specifyintelligent-controller operation. The time periods spanned by variousdifferent types of control schedules and the granularity, in time, ofcontrol schedules may vary widely depending on the control tasks forwhich particular controllers are designed.

FIG. 25A shows the highest-level intelligent-controller control logic.This high-level control logic comprises an event-handling loop in whichvarious types of control-related events are handled by the intelligentcontroller. In FIG. 25A, four specific types of control-related eventsare handled, but, in general, the event-handling loop may handle manyadditional types of control-related events that occur at lower levelswithin the intelligent-controller logic. Examples include communicationsevents, in which the intelligent controller receives or transmits datato remote entities, such as remote smart-home devices andcloud-computing servers. Other types of control-related events includecontrol-related events related to system activation and deactivationaccording to observed parameters and control schedules, various types ofalarms and timers that may be triggered by sensor data falling outsideof control-schedule-specified ranges for detection and unusual or rareevents that require specialized handling. Rather than attempt todescribe all the various different types of control-related events thatmay be handled by an intelligent controller, FIG. 25A illustrateshandling of four example control-related events.

In step 2502, the intelligent controller waits for a nextcontrol-related event to occur. When a control-related event occurs,control flows to step 2504, and the intelligent controller determineswhether an immediate-control input has been input by a user or remoteentity through the immediate-control-input interface. When animmediate-control input has been input by a user or other remote entity,as determined in step 2504, the intelligent controller carries out theimmediate control input, in step 2505, generally by changing internallystored specified ranges for parameter values and, when needed,activating one or more controlled entities, and then theimmediate-control input is recorded in memory, in step 2506. When anadditional setpoint or other schedule feature needs to be added toterminate the immediate-control input, as determined in step 2507, thenthe additional setpoint or other schedule feature is added to thecontrol schedule, in step 2508. Examples of such added setpoints arediscussed above with reference to FIGS. 15A-G. When the control-relatedevent that triggered exit from step 2502 is a timer event indicatingthat the current time is that of a scheduled setpoint or scheduledcontrol, as determined in step 2509, then the intelligent controllercarries out the scheduled controller setpoint in step 2510. When thescheduled control carried out in step 2510 is a temporary scheduledcontrol added in step 2508 to terminate an immediate-control input, asdetermined in step 2511, then the temporary scheduled control is deletedin step 2512. When the control-related event that triggered exit fromstep 2502 is a change made by a user or remote entity to the controlschedule via the control-schedule-change interface, as determined instep 2513, then, when the schedule change is prospective, as determinedin step 1514, the schedule change is made by the intelligent controllerto the existing control schedule in step 2515, as discussed above withreference to FIGS. 18A-B. Otherwise, the schedule change isretrospective, and is recorded by the intelligent controller into memoryin step 2516 for later use in varying a new provisional schedule at thetermination of the current monitoring period.

When the control-related event that triggered exit from 2502 is a timerevent associated with the end of the current monitoring period, asdetermined in step 2517, then a monitoring-period routine is called, instep 2518, to process recorded immediate-control inputs and schedulechanges, as discussed above with reference to FIGS. 15A-24F. Whenadditional control-related events occur after exit from step 2502, whichare generally queued to an occurred event queue, as determined in step2519, control flows back to step 2504 for handling a next queued event.Otherwise, control flows back in step 2502 where the intelligentcontroller waits for a next control-related event.

FIG. 25B provides a control-flow diagram for the routine “monitoringperiod” called in step 2518 in FIG. 25A. In step 2522, the intelligentcontroller accesses a state variable that stores an indication of thecurrent learning mode. When the current learning mode is anaggressive-learning mode, as determined in step 2523, the routine“aggressive monitoring period” is called in step 2524. Otherwise, theroutine “steady-state monitoring period” is called, in step 2525. Whilethis control-flow diagram is simple, it clearly shows the feature ofautomated-control-schedule-learning discussed above with reference toFIGS. 11D-E and FIG. 12. Automated-control-schedule learning isbifurcated into an initial, aggressive-learning period followed by asubsequent steady-state learning period.

FIG. 25C provides a control-flow diagram for the routine “aggressivemonitoring period” called in step 2524 of FIG. 25B. This routine iscalled at the end of each monitoring period. In the example discussedabove, a monitoring period terminates at the end of each daily controlschedule, immediately after 12:00 p.m. However, monitoring periods, inalternative implementations, may occur at a variety of other differenttime intervals and may even occur variably, depending on othercharacteristics and parameters. Monitoring periods are generally thesmallest-granularity time periods corresponding to control schedules orsub-schedules, as discussed above.

In step 2527, the intelligent controller combines all recordedimmediate-control inputs with the existing control schedule, asdiscussed above with reference to FIGS. 16B and 20A. In step 2528, theroutine “cluster” is called in order to partition the recordedimmediate-control inputs and schedule changes andexisting-control-schedule setpoints to clusters, as discussed above withreference to FIGS. 16C-D and FIG. 20B. In step 2529, the intelligentcontroller calls the routine “simplify clusters” to resolve the varioussetpoints within each cluster, as discussed above with reference toFIGS. 16A-20C. In step 2530, the intelligent controller calls theroutine “generate new schedule” to generate a new provisional schedulefollowing cluster resolution, as discussed above with reference to FIGS.20C and 21A-B. In step 2531, the intelligent controller calls theroutine “propagateNewSchedule” discussed above with reference to FIGS.22A-24I, in order to propagate features of the provisional schedulegenerated in step 2530 to related sub-schedules and control schedules ofthe intelligent controller's control schedule. In step 2532, theintelligent controller determines whether or not the currently completedmonitoring period is the final monitoring period in theaggressive-learning mode. When the recently completed monitoring periodis the final monitoring period in the aggressive-monitoring learningmode, as determined in step 2532, then, in step 2533, the intelligentcontroller sets various state variables that control the currentlearning mode to indicate that the intelligent controller is nowoperating in the steady-state learning mode and, in step 2534, setsvarious learning parameters to parameter values compatible with phase Iof steady-state learning.

Many different learning parameters may be used in differentimplementations of automated control-schedule learning. In the currentlydiscussed implementation, learning parameters may include the amount oftime that immediate-control inputs are carried out before termination bythe intelligent controller, the magnitudes of the various threshold Δtand threshold ΔP values used in cluster resolution and resolution ofpropagated setpoints with respect to existing control schedules.Finally, in step 2535, the recorded immediate-control inputs andschedule changes, as well as clustering information and other temporaryinformation derived and stored during creation of a new provisionalschedule and propagation of the provisional schedule are deleted and thelearning logic is reinitialized to begin a subsequent monitoring period.

FIG. 25D provides a control-flow diagram for the routine “cluster”called in step 2528 of FIG. 25C. In step 2537, the local variable Δtintis set to a learning-mode and learning-phase-dependent value. Then, inthe while-loop of steps 2538-2542, the routine “interval cluster” isrepeatedly called in order to generate clusters within the existingcontrol schedule until one or more clustering criteria are satisfied, asdetermined in step 2540. Prior to satisfaction of the clusteringcriteria, the value of Δtint is incremented, in step 2542, prior to eachnext call to the routine “interval cluster” in step 2539, in order toalter the next clustering for satisfying the clustering criteria. Thevariable Δtint corresponds to the minimum length of time betweensetpoints that results in the setpoints being classified as belonging totwo different clusters, as discussed above with reference to FIG. 16C,or the time period 1610 the interval between two clusters. DecreasingΔtint generally produces additional clusters.

Various different types of clustering criteria may be used by anintelligent controller. In general, it is desirable to generate asufficient number of clusters to produce adequate control-schedulesimplification, but too many clusters result in additionalcontrol-schedule complexity. The clustering criteria are designed,therefore, to choose a Δtint sufficient to produce a desirable level ofclustering that leads to a desirable level of control-schedulesimplification. The while-loop continues while the value of Δtintremains within an acceptable range of values. When the clusteringcriteria fails to be satisfied by repeated calls to the routine“intervalCluster” in the while-loop of steps 2538-2542, then, in step2543, one or more alternative clustering methods may be employed togenerate clusters, when needed for control-schedule simplification.Alternative methods may involve selecting clusters based on localmaximum and minimum parameter values indicated in the control scheduleor, when all else fails, by selecting, as cluster boundaries, a numberof the longest setpoint-free time intervals within the setpointsgenerated in step 2537.

FIG. 25E provides a control-flow diagram for the routine “intervalcluster” called in step 2539 of FIG. 25D. In step 2545, the intelligentcontroller determines whether or not a setpoint coincides with thebeginning time of the control schedule corresponding to the monitoringperiod. When a setpoint does coincide with the beginning of the time ofthe control schedule, as determined in step 2545, then the localvariable “startCluster” is set to the start time of the control scheduleand the local variable “numCluster” is set to 1, in step 2546.Otherwise, the local variable “numCluster” is set to 0 in step 2547. Instep 2548, the local variable “lastSP” is set to the start time of thecontrol schedule and the local variable “curT” is set to “lastSP” plus atime increment Δtinc in step 2548. The local variable “curt” is anindication of the current time point in the control schedule beingconsidered, the local variable “numCluster” is an indication of thenumber of setpoints in a next cluster that is being created, the localvariable “startCluster” is an indication of the point in time of thefirst setpoint in the cluster, and the local variable “lastSP” is anindication of the time of the last detected setpoint in the controlschedule. Next, in the while-loop of steps 2549-2559, the controlschedule corresponding to the monitoring period is traversed, from startto finish, in order to generate a sequence of clusters from the controlschedule. In step 2550, a local variable Δt is set to the length of thetime interval between the last detected setpoint and the current pointin time that is being considered. When there is a setpoint thatcoincides with the current point in time, as determined in step 2551,then a routine “nextSP” is called, in step 2552, to consider and processthe setpoint. Otherwise, when Δt is greater than Δtint, as determined instep 2553, then, when a cluster is being processed, as determined instep 2554, the cluster is closed and stored, in step 2555, and the localvariable “numCluster” is reinitialized to begin processing of a nextcluster. The local variable “curt” is incremented, in step 2556, and thewhile-loop continues to iterate when curT is less than or equal to thetime at which the control schedule ends, as determined in step 2557.When the while-loop ends, and when a cluster was being created, asdetermined in step 2558, then that cluster is closed and stored in step2559.

FIG. 25F provides a control-flow diagram for the routine “next SP”called in step 2552 of FIG. 25E. In step 2560, the intelligentcontroller determines whether or not a cluster was being created at thetime of the routine call. When a cluster was being created, and when Δtis less than Δtint, as determined in step 2561, then the currentsetpoint is added to the cluster in step 2562. Otherwise, the currentlyconsidered cluster is closed and stored, in step 2563. When a clusterwas not being created, then the currently detected setpoint becomes thefirst setpoint in a new cluster, in step 2564.

FIG. 25G provides a control-flow diagram for the routine “simplifyclusters” called in step 2529 of FIG. 25C. This routine is a simplefor-loop, comprising steps 2566-2568 in which each cluster, determinedby the routine “cluster” called in step 2528 of FIG. 25C, is simplified,as discussed above with reference to FIGS. 16A-21D. The cluster issimplified by a call to the routine “simplify” in step 2567.

FIG. 25H is a control-flow diagram for the routine “simplify” called instep 2567 of FIG. 25G. In step 2570, the intelligent controllerdetermines whether or not the currently considered cluster contains anyschedule-change setpoints. When the currently considered clustercontains schedule-change setpoints, then any immediate-control setpointsare removed, in step 2572. When the cluster contains only a singleschedule-change setpoint, as determined in step 2573, then that singleschedule-change setpoint is left to represent the entire cluster, instep 2574. Otherwise, the multiple schedule changes are resolved intozero, one, or two setpoints to represent the cluster as discussed abovewith reference to FIGS. 17A-E in step 2575. The zero, one, or twosetpoints are then entered into the existing control schedule in step2576. When the cluster does not contain any schedule-change setpoints,as determined in step 2570, and when the setpoints in the cluster can bereplaced by a single setpoint, as determined in step 2577, as discussedabove with reference to FIGS. 17A and 17C, then the setpoints of thecluster are replaced with a single setpoint, in step 2578, as discussedabove with reference to FIGS. 17A and 17C. Note that, as discussed abovewith reference to FIGS. 20A-C, the setpoints are associated with labels“s” and “i” to indicate whether they are derived from scheduledsetpoints or from immediate-control setpoints. Similarly, when thesetpoints of the cluster can be replaced by two setpoints, as determinedin step 2579, then the cluster is replaced by the two setpoints withappropriate labels, as discussed above with reference to FIGS. 17D-E, instep 2580. Otherwise, the condition described with reference to FIG.717B has occurred, in which case all of the remaining setpoints aredeleted from the cluster in step 2581.

FIG. 25I provides a control-flow diagram for the routine “generate newschedule” called in step 2530 of FIG. 25C. When the new provisionalschedule includes two or more immediate-control setpoints, as determinedin step 2583, then the routine “spread” is called in step 2584. Thisroutine spreads “i”-labeled setpoints, as discussed above to FIGS.21A-B. The control schedule is then stored as a new current controlschedule for the time period in step 2585 with the indications ofwhether the setpoints are derived from immediate-control setpoints orschedule setpoints retained for a subsequent propagation step in step2586.

FIG. 25J provides a control-flow diagram for the routine “spread,”called in step 2584 in FIG. 25I. In step 2587, the local variable“first” is set to the first immediate-control setpoint in theprovisional schedule. In step 2588, the variable “second” is set to thesecond immediate-control setpoint in the provisional schedule. Then, inthe while-loop of steps 2589-2599, the provisional schedule is traversedin order to detect pairs of immediate-control setpoints that are closertogether, in time, than a threshold length of time Δt1. The secondsetpoint is moved, in time, in steps 2592-2596, either by a fixed timeinterval Δts or to a point halfway between the previous setpoint and thenext setpoint, in order to spread the immediate-control setpoints apart.

FIG. 25K provides a control-flow diagram for the routine “propagate newschedule” called in step 2531 of FIG. 25C. This routine propagates aprovisional schedule created in step 2530 in FIG. 25C to relatedsub-schedules, as discussed above with reference to FIGS. 22A-B. In step2599 a, the intelligent controller determines the additionalsub-schedules or schedules to which the provisional schedule generatedin step 2530 should be propagated. Then, in the for-loop of steps 2599b-2599 e, the retained immediate-control setpoints, retained in step2586 in FIG. 25I, are propagated to a next related control schedule andthose setpoints, along with existing-control-schedule setpoints in thenext control schedule, are resolved by a call to the routine “resolveadditional schedule,” in step 2599 d.

FIG. 25L provides a control-flow diagram for the routine “resolveadditional schedule,” called in step 2599 d of FIG. 25K. In step 2599 f,the intelligent controller accesses a stored set of schedule-resolutionrules, such as those discussed above with reference to FIGS. 24A-I, andsets the local variable j to the number of schedule-resolution rules tobe applied. Again, in the nested for-loops of steps 2599 g-2599 n, therules are applied to each immediate-control setpoint in the set ofsetpoints generated in step 2599 c of FIG. 25K. The rules are applied insequence to each immediate-control setpoint until either the setpoint isdeleted, as determined in step 2599 j, or until the rule is successfullyapplied to simplify the schedule, in step 2599 k. Once all thepropagated setpoints have been resolved in the nested for-loops of steps2599 g-2599 n, then the schedule is stored as a new provisionalschedule, in step 2599 o.

FIG. 25M provides a control-flow diagram for the routine “steady-statemonitoring” called in step 2525 of FIG. 25B. This routine is similar tothe routine “aggressive monitoring period” shown in FIG. 25C and calledin step 2524 of FIG. 25B. Many of the steps are, in fact, nearlyidentical, and will not be again described, in the interest of brevity.However, step 2599 q is an additional step not present in the routine“aggressiveMonitoringPeriod.” In this step, the immediate-controlsetpoints and schedule-change setpoints overlaid on theexisting-control-schedule setpoints are used to search a database ofrecent historical control schedules in order to determine whether or notthe set of setpoints is more closely related to another control scheduleto which the intelligent controller is to be targeted or shifted. Whenthe control-schedule shift is indicated by this search, as determined instep 2599 h, then the shift is carried out in step 2599I, and the storedimmediate-controls and schedule changes are combined with a sub-scheduleof the target schedule to which the intelligent controller is shifted,in step 2599 t, prior to carrying out generation of the new provisionalschedule. The historical-search routine, called in step 2599 q, may alsofilter the recorded immediate-control setpoints and schedule-changesetpoints recorded during the monitoring period with respect to one ormore control schedules or sub-schedules corresponding to the monitoringperiod. This is part of a more conservative learning approach, asopposed to the aggressive learning approach used in theaggressive-learning mode, that seeks to only conservatively alter acontrol schedule based on inputs recorded during a monitoring period.Thus, while the procedure carried out at the end of a monitoring periodare similar both for the aggressive-learning mode and the steady-statelearning mode, schedule changes are carried out in a more conservativefashion during steady-state learning, and the schedule changes becomeincreasingly conservative with each successive phase of steady-statelearning. With extensive recent and historical control-scheduleinformation at hand, the intelligent controller can make intelligent andincreasingly accurate predictions of whether immediate-control inputsand schedule changes that occurred during the monitoring period reflectthe user's desire for long-term changes to the control schedule or,instead, reflect temporary control changes related to temporally localevents and conditions.

As mentioned above, an intelligent controller may employ multipledifferent control schedules that are applicable over different periodsof time. For example, in the case of a residential HVAC thermostatcontroller, an intelligent controller may use a variety of differentcontrol schedules applicable to different seasons during the year;perhaps a different control schedule for winter, summer, spring, andfall. Other types of intelligent controllers may use a number of controlschedules for various different periods of control that span minutes andhours to months, years, and even greater periods of time.

FIG. 26 illustrates three different week-based control schedulescorresponding to three different control modes for operation of anintelligent controller. Each of the three control schedules 2602-2604 isa different week-based control schedule that controlsintelligent-controller operation for a period of time until operationalcontrol is shifted, in step 2599 s of FIG. 25M, to another of thecontrol schedules. FIG. 27 illustrates a state-transition diagram for anintelligent controller that operates according to seven differentcontrol schedules. The modes of operation controlled by the particularcontrol schedules are shown as disks, such as disk 2702, and thetransitions between the modes of operation are shown as curved arrows,such as curved arrow 2704. In the case shown in FIG. 27, thestate-transition diagram expresses a deterministic, higher-level controlschedule for the intelligent controller comprising seven differentoperational modes, each controlled by a particular control schedule.Each of these particular control schedules may, in turn, be composed ofadditional hierarchical levels of sub-schedules. The automated-learningmethods to which the current application is directed can accommodateautomated learning of multiple control schedules and sub-schedules,regardless of their hierarchical organization. Monitoring periodsgenerally encompass the shortest-time, smallest-grain sub-schedules in ahierarchy, and transitions between sub-schedules and higher-levelcontrol schedules are controlled by higher-level control schedules, suchas the transition-state-diagram-expressed higher-level control scheduleillustrated in FIG. 27, by the sequential ordering of sub-scheduleswithin a larger control schedule, such as the daily sub-schedules withina weekly control schedule discussed with reference to FIG. 13, oraccording to many other control-schedule organizations andschedule-shift criteria.

FIGS. 28A-C illustrate one type of control-schedule transition that maybe carried out by an intelligent controller. FIG. 28A shows the existingcontrol schedule according to which the intelligent controller iscurrently operating. FIG. 28B shows recorded immediate-control inputsover a recently completed monitoring period superimposed over thecontrol schedule shown in FIG. 28A. These immediate-control inputs2802-2805 appear to represent a significant departure from the existingcontrol schedule 2800. In step 2599 q of FIG. 25M, an intelligentcontroller may consider various alternative control schedules orhistorical control schedules, including control schedule 2810, shown inFIG. 28C, that may be alternate control schedules for the recentlycompleted monitoring period. As it turns out, resolution of theimmediate-control inputs with the existing control schedule wouldproduce a control schedule very close to control schedule 2810 shown inFIG. 28C. This then provides a strong indication to the intelligentcontroller that the recorded immediate-control inputs may suggest a needto shift control to control schedule 2810 rather than to modify theexisting control schedule and continue using the modified controlschedule. Although this is one type of schedule-change transition thatmay occur in step 2599 s in FIG. 25M, other schedule-change shifts maybe controlled by knowledge of the current date, day of the week, andperhaps knowledge of various environmental parameters that togetherspecify use of multiple control schedules to be used to controlintelligent-control operations.

FIGS. 29-30 illustrate types of considerations that may be made by anintelligent controller during steady-state-learning phases. In FIG. 29,the plot of a new provisional schedule 2902 is shown, along with similarplots of 15 recent or historical control schedules or provisionalschedules applicable to the same time period 2904-2918. Visualcomparison of the new provisional schedule 2902 to the recent andhistorical provisional schedules 2904-2918 immediately reveals that thenew provisional schedule represents a rather radical change in thecontrol regime. During steady-state learning, such radical changes maynot be propagated or used to replace existing control schedules, but mayinstead be recorded and used for propagation or replacement purposesonly when the accumulated record of recent and historical provisionalschedules provide better support for considering the provisionalschedule as an indication of future user intent. For example, as shownin FIG. 30, were the new provisional schedule compared to a record ofrecent and/or historical control schedules 3002-3016, the intelligentcontroller would be far more likely to use new provisional schedule 2902for replacement or propagation purposes.

Automated Schedule Learning in the Context of an Intelligent Thermostat

An implementation of automated control-schedule learning is included ina next-described intelligent thermostat. The intelligent thermostat isprovided with a selectively layered functionality that exposesunsophisticated users to a simple user interface, but provides advancedusers with an ability to access and manipulate many differentenergy-saving and energy tracking capabilities. Even for the case ofunsophisticated users who are only exposed to the simple user interface,the intelligent thermostat provides advanced energy-saving functionalitythat runs in the background. The intelligent thermostat usesmulti-sensor technology to learn the heating and cooling environment inwhich the intelligent thermostat is located and to optimizeenergy-saving settings.

The intelligent thermostat also learns about the users, beginning with asetup dialog in which the user answers a few simple questions, and thencontinuing, over time, using multi-sensor technology to detect useroccupancy patterns and to track the way the user controls thetemperature using schedule changes and immediate-control inputs. On anongoing basis, the intelligent thermostat processes the learned andsensed information, automatically adjusting environmental controlsettings to optimize energy usage while, at the same time, maintainingthe temperature within the environment at desirable levels, according tothe learned occupancy patterns and comfort preferences of one or moreusers.

Advantageously, the selectively layered functionality of the intelligentthermostat allows for effective operation in a variety of differenttechnological circumstances within home and business environments. Forsimple environments having no wireless home network or Internetconnectivity, the intelligent thermostat operates effectively in astandalone mode, learning and adapting to an environment based onmulti-sensor technology and user input. However, for environments thathave home network or Internet connectivity, the intelligent thermostatoperates effectively in a network-connected mode to offer additionalcapabilities.

When the intelligent thermostat is connected to the Internet via a homenetwork, such as through IEEE 802.11 (Wi-Fi) connectivity, theintelligent thermostat may: (1) provide real-time or aggregated homeenergy performance data to a utility company, intelligent thermostatdata service provider, intelligent thermostats in other homes, or otherdata destinations; (2) receive real-time or aggregated home energyperformance data from a utility company, intelligent thermostat dataservice provider, intelligent thermostats in other homes, or other datasources; (3) receive new energy control instructions and/or otherupgrades from one or more intelligent thermostat data service providersor other sources; (4) receive current and forecasted weather informationfor inclusion in energy-saving control algorithm processing; (5) receiveuser control commands from the user's computer, network-connectedtelevision, smart phone, and/or other stationary or portable datacommunication appliance; (6) provide an interactive user interface to auser through a digital appliance; (7) receive control commands andinformation from an external energy management advisor, such as asubscription-based service aimed at leveraging collected informationfrom multiple sources to generate energy-saving control commands and/orprofiles for their subscribers; (8) receive control commands andinformation from an external energy management authority, such as autility company to which limited authority has been voluntarily given tocontrol the intelligent thermostat in exchange for rebates or other costincentives; (9) provide alarms, alerts, or other information to a useron a digital appliance based on intelligent thermostat-sensedHVAC-related events; (10) provide alarms, alerts, or other informationto the user on a digital appliance based on intelligentthermostat-sensed non-HVAC related events; and (11) provide a variety ofother useful functions enabled by network connectivity.

FIG. 31A illustrates a perspective view of an intelligent thermostat.The intelligent thermostat 3100 has a sleek, elegant appearance. Theintelligent thermostat 3100 comprises a circular main body 3108 with adiameter of about 8 cm and that has a visually pleasing outer finish,such as a satin nickel or chrome finish. A cap-like structure comprisinga rotatable outer ring 3106, a sensor ring 3104, and a circular displaymonitor 3102 is separated from the main body 3108 by a small peripheralgap 3110. The outer ring 3106 may have an outer finish identical to thatof the main body 3108, while the sensor ring 3104 and circular displaymonitor 3102 may have a common circular glass (or plastic) outercovering that is gently arced in an outward direction and that providesa sleek yet solid and durable-looking overall appearance. The sensorring 3104 contains any of a wide variety of sensors, including infraredsensors, visible-light sensors, and acoustic sensors. The glass orplastic that covers the sensor ring 3104 may be smoked or mirrored suchthat the sensors themselves are not visible to the user. An air ventingfunctionality may be provided, via the peripheral gap 3110, which allowsthe ambient air to be sensed by the internal sensors without the needfor gills or grill-like vents.

FIGS. 31B-31C illustrate the intelligent thermostat being controlled bya user. The intelligent thermostat 3100 is controlled by two types ofuser input: (1) a rotation of the outer ring 3106 (FIG. 31B); and (2) aninward push on the outer ring 3106 (FIG. 31C) until an audible and/ortactile “click” occurs. The inward push may cause the outer ring 3106 tomove forward, while in another implementation, the entire cap-likestructure, including both the outer ring 3106 and the glass covering ofthe sensor ring 3104 and circular display monitor 3102, move inwardlytogether when pushed. The sensor ring 3104, the circular display monitor3102, and the common glass covering do not rotate with outer ring 3106in one implementation.

By rotation of the outer ring 3106, or ring rotation, and inward pushingof the outer ring 3106, or inward click, the intelligent thermostat 3100can receive all necessary information from the user for basic setup andoperation. The outer ring 3106 is mechanically mounted in a manner thatprovides a smooth yet viscous feel to the user, for further promoting anoverall feeling of elegance while also reducing spurious or unwantedrotational inputs. The intelligent thermostat 3100 recognizes threefundamental user inputs: (1) ring rotate left, (2) ring rotate right,and (3) inward click. In other implementations, more complex fundamentaluser inputs can be recognized, such as double-click or triple-clickinward presses and speed-sensitive or acceleration-sensitive rotationalinputs.

FIG. 32 illustrates an exploded perspective view of the intelligentthermostat and an HVAC-coupling wall dock. The HVAC-coupling wall dock3206 has the functionality as a very simple, elemental, standalonethermostat when the intelligent thermostat 3100 is removed, theelemental thermostat including a standard temperature readout/settingdial 3208 and a simple COOL-OFF-HEAT switch 3209. This can prove usefulfor a variety of situations, such as when the intelligent thermostat3100 needs to be removed for service or repair for an extended period oftime. In one implementation, the elemental thermostat components 3208and 3209 are entirely mechanical in nature, so that no electrical poweris needed to trip the control relays. In other implementations, simpleelectronic controls, such as electrical up/down buttons and/or an LCDreadout, are provided. In other implementations, a subset of theadvanced functionalities of the intelligent thermostat 3100 can beprovided, such as elemental network access to allow remote control thatprovides a brain-stem functionality while the intelligent thermostat istemporarily removed.

FIGS. 33A-33B illustrate exploded front and rear perspective views ofthe intelligent thermostat. FIGS. 33A-33B show the intelligentthermostat 3302 with respect to its two main components: (1) the headunit 3204; and (2) the back plate 3306. In the drawings shown, the zdirection is outward from the wall, the y direction is the head-to-toedirection relative to a walk-up user, and the x direction is the user'sleft-to-right direction.

FIGS. 34A-34B illustrate exploded front and rear perspective views,respectively, of the head unit. Head unit 3304 includes a head unitframe 3310, the outer ring 3311, a head unit frontal assembly 3312, afront lens 3313, and a front grille 3314. Electrical components on thehead unit frontal assembly 3312 can connect to electrical components onthe backplate 3306 via ribbon cables and/or other plug type electricalconnectors.

FIGS. 35A-35B illustrate exploded front and rear perspective views,respectively, of the head unit frontal assembly. Head unit frontalassembly 3312 comprises a head unit circuit board 3316, a head unitfront plate 3318, and an LCD module 3322. The components of the frontside of head unit circuit board 3316 are hidden behind an RF shield inFIG. 35A. A rechargeable Lithium-Ion battery 3325 is located on the backof the head unit circuit board 3316, which, in one implementation, has anominal voltage of 3.7 volts and a nominal capacity of 560 mAh. Toextend battery life, the battery 3325 is normally not charged beyond 450mAh by the thermostat battery charging circuitry. Moreover, although thebattery 3325 is rated to be capable of being charged to 4.2 volts, theintelligent thermostat battery-charging circuitry normally does notcharge the intelligent thermostat beyond 3.95 volts. Also shown in FIG.35B is an optical finger navigation module 3324 that is configured andpositioned to sense rotation of the outer ring 3311. The module 3324uses methods analogous to the operation of optical computer mice tosense the movement of a texturable surface on a facing periphery of theouter ring 3311. Notably, the module 3324 is one of the very few sensorsthat are controlled by the relatively power-intensive head unitmicroprocessor rather than the relatively low-power backplatemicroprocessor. This is achievable without excessive power drain becausethe head unit microprocessor is already awake when a user is manuallyturning the dial, avoiding excessive wake-up power drain.Advantageously, very fast response can also be provided by the head unitmicroprocessor. Also shown in FIG. 35A is a Fresnel lens 3320 thatoperates in conjunction with a PIR motion sensor disposesthereunderneath.

FIGS. 36A-36B illustrate exploded front and rear perspective views,respectively, of the backplate unit. Backplate unit 3306 comprises abackplate rear plate 3330, a backplate circuit board 3332, and abackplate cover 3339. FIG. 36A shows the HVAC wire connectors 3334 thatinclude integrated wire-insertion-sensing circuitry and two relativelylarge capacitors 3336 that are used by the power stealing circuitry thatis mounted on the back side of the backplate circuit board 3332.

FIG. 37 shows a perspective view of a partially assembled head unit. Incertain implementations, placement of grille member 3314 over theFresnel lens 3320 and an associated PIR motion sensor 3344 conceals andprotects these PIR sensing elements, while horizontal slots in thegrille member 3314 allow the PIR motion sensing hardware, despite beingconcealed, to detect the lateral motion of occupants in a room or area.A temperature sensor 3340 uses a pair of thermal sensors to moreaccurately measure ambient temperature. A first or upper thermal sensor3341 associated with temperature sensor 3340 gathers temperature datacloser to the area outside or on the exterior of the thermostat while asecond or lower thermal sensor 3342 collects temperature data moreclosely associated with the interior of the housing. In oneimplementation, each of the temperature sensors 3341 and 3342 comprisesa Texas Instruments TMP112 digital temperature sensor chip, while thePIR motion sensor 3344 comprises PerkinElmer DigiPyro PYD 1998dual-element pyrodetector.

To more accurately determine the ambient temperature, the temperaturetaken from the lower thermal sensor 3342 is considered in conjunctionwith the temperatures measured by the upper thermal sensor 3341 and whendetermining the effective ambient temperature. This configuration can beused to compensate for the effects of internal heat produced in thethermostat by the microprocessor(s) and/or other electronic components,obviating or minimizing temperature measurement errors that mightotherwise be suffered. In some implementations, the accuracy of theambient temperature measurement may be further enhanced by thermallycoupling upper thermal sensor 3341 of temperature sensor 3340 to grillemember 3314 as the upper thermal sensor 3341 better reflects the ambienttemperature than lower thermal sensor 3342.

FIG. 38 illustrates the head unit circuit board. The head unit circuitboard 3316 comprises a head unit microprocessor 3802 (such as a TexasInstruments AM3703 chip) and an associated oscillator 3804, along withDDR SDRAM memory 3806, and mass NAND storage 3808. A Wi-Fi module 3810,such as a Murata Wireless Solutions LBWA19XSLZ module, which is based onthe Texas Instruments WL1270 chipset supporting the 802.11b/g/n WLANstandard, is provided in a separate compartment of RF shielding 3834 forWi-Fi capability. Wi-Fi module 3810 is associated with supportingcircuitry 3812 including an oscillator 3814. A ZigBee module 3816, whichcan be, for example, a C2530F256 module from Texas Instruments, isprovided, also in a separately shielded RF compartment, for ZigBeecapability. The ZigBee module 3816 is associated with supportingcircuitry 3818, including an oscillator 3819 and a low-noise amplifier3820. Display backlight voltage conversion circuitry 3822, piezoelectricdriving circuitry 3824, and power management circuitry 3826 areadditionally provided. A proximity sensor and an ambient light sensor(PROX/ALS), more particularly a Silicon Labs SI1142 Proximity/AmbientLight Sensor with an I2C Interface, is provided on a flex circuit 3828that attaches to the back of the head unit circuit board by a flexcircuit connector 3830. Battery-charging-supervision-disconnectcircuitry 3832 and spring/RF antennas 3836 are additionally provided. Atemperature sensor 3838 and a PIR motion sensor 3840 are additionallyprovided.

FIG. 39 illustrates a rear view of the backplate circuit board. Thebackplate circuit board 3332 comprises a backplateprocessor/microcontroller 3902, such as a Texas Instruments MSP430FSystem-on-Chip Microcontroller that includes an on-board memory 3903.The backplate circuit board 3332 further comprises power-supplycircuitry 3904, which includes power-stealing circuitry, and switchcircuitry 3906 for each HVAC respective HVAC function. For each suchfunction, the switch circuitry 3906 includes an isolation transformer3908 and a back-to-back NFET package 3910. The use of FETs in theswitching circuitry allows for active power stealing, i.e., taking powerduring the HVAC ON cycle, by briefly diverting power from the HVAC relaycircuit to the reservoir capacitors for a very small interval, such as100 micro-seconds. This time is small enough not to trip the HVAC relayinto the OFF state but is sufficient to charge up the reservoircapacitors. The use of FETs allows for this fast switching time (100micro-seconds), which would be difficult to achieve using relays (whichstay on for tens of milliseconds). Also, such relays would readilydegrade with fast switching, and they would also make audible noise. Incontrast, the FETS operate with essentially no audible noise. A combinedtemperature/humidity sensor module 3912, such as a Sensirion SHT21module, is additionally provided. The backplate microcontroller 3902performs polling of the various sensors, sensing for mechanical wireinsertion at installation, alerting the head unit regarding current vs.setpoint temperature conditions and actuating the switches accordingly,and other functions such as looking for appropriate signal on theinserted wire at installation.

Next, an implementation of the above-describedautomated-control-schedule-learning methods for the above-describedintelligent thermostat is provided. FIGS. 40A-D illustrate steps forachieving initial learning. FIGS. 41A-M illustrate a progression ofconceptual views of a thermostat schedule. The progression of conceptualviews of a thermostat schedule occurs as processing is performedaccording to selected steps of FIGS. 40A-40D, for an example one-daymonitoring period during an initial aggressive-learning period. For oneimplementation, the steps of FIGS. 40A-40D are carried out by the headunit microprocessor of thermostat 3302, with or without Internetconnectivity. In other implementations, one or more of the steps ofFIGS. 40A-40D can be carried out by a cloud server to which thethermostat 3302 has network connectivity. While the example presented inFIGS. 41A-41M is for a heating schedule scenario, the described methodis likewise applicable for cooling-schedule learning, and can be readilyextended to HVAC schedules containing mixtures of heating setpoints,cooling setpoints, and/or range setpoints. While the examples of FIGS.40A-41M are presented in the particular context of establishing a weeklyschedule, which represents one particularly appropriate time basis forHVAC schedule establishment and execution, in other implementations abi-weekly HVAC schedule, a semi-weekly HVAC schedule, a monthly HVACschedule, a bi-monthly HVAC schedule, a seasonal HVAC schedule, andother types of schedules may be established. While the examples of FIGS.40A-41M are presented and/or discussed in terms of a typical residentialinstallation, this is for the purpose of clarity of explanation. Themethods are applicable to a wide variety of other types of enclosures,such as retail stores, business offices, industrial settings, and soforth. In the discussion that follows, the time of a particular useraction or setpoint entry are generally expressed as both the day and thetime of day of that action or entry, while the phrase “time of day” isgenerally used to express a particular time of day.

The initial learning process represents an “aggressive learning”approach in which the goal is to quickly establish an at least roughlyappropriate HVAC schedule for a user or users based on a very briefperiod of automated observation and tracking of user behavior. Once theinitial learning process is established, the thermostat 3302 thenswitches over to steady-state learning, which is directed to perceivingand adapting to longer-term repeated behaviors of the user or users. Inmost cases, the initial learning process is begun, in step 4002, inresponse to a new installation and startup of the thermostat 3302 in aresidence or other controlled environment, often following auser-friendly setup interview. Initial learning can also be invoked byother events, such as a factory reset of the intelligent thermostat 3302or an explicit request of a user who may wish for the thermostat 3302 torepeat the aggressive-learning phase.

In step 4004, a default beginning schedule is accessed. For oneimplementation, the beginning schedule is simply a single setpoint thattakes effect at 8 AM each day and that includes a single setpointtemperature. This single setpoint temperature is dictated by a userresponse that is provided near the end of the setup interview or uponinvocation of initial learning, where the user is asked whether to startlearning a heating schedule or a cooling schedule. When the user choosesheating, the initial single setpoint temperature is set to 68° F., orsome other appropriate heating setpoint temperature, and when the userchooses cooling, the initial single setpoint temperature is set to 80°F., or some other appropriate cooling setpoint temperature. In otherimplementations, the default beginning schedule can be one of aplurality of predetermined template schedules that ° is selecteddirectly or indirectly by the user at the initial setup interview. FIG.41A illustrates an example of a default beginning schedule havingheating setpoints labeled “a” through “g”.

In step 4006, a new monitoring period is begun. The selection of aone-day monitoring period has been found to provide good results in thecase of control-schedule acquisition in an intelligent thermostat.However, other monitoring periods can be used, including multi-dayblocks of time, sub-day blocks of time, other suitable periods, and canalternatively be variable, random, or continuous. For example, whenperformed on a continuous basis, any user setpoint change or scheduledsetpoint input can be used as a trigger for processing that informationin conjunction with the present schedule to produce a next version,iteration, or refinement of the schedule. For one implementation, inwhich the thermostat 3302 is a power-stealing thermostat having arechargeable battery, the period of one day has been found to provide asuitable balance between the freshness of the schedule revisions and theneed to maintain a modest computing load on the head unit microprocessorto preserve battery power.

In step 4008, throughout the day, the intelligent thermostat 3302receives and stores both immediate-control and schedule-change inputs.FIG. 41B shows a representation of a plurality of immediate-control andschedule-change user setpoint entries that were made on a typical day ofinitial learning, which happens to be a Tuesday in the currentlydescribed example. In the following discussion and in the accompanyingdrawings, including FIGS. 41A-41M, a preceding superscript “N”identifies a schedule-change, or non-real-time (“NRT”), setpoint entryand a preceding superscript “R” identifies an immediate-control, orreal-time (“RT”) setpoint entry. An encircled number represents apre-existing scheduled setpoint. For each NRT setpoint, a succeedingsubscript that identifies the entry time of that NRT setpoint is alsoprovided. No such subscript is needed for RT setpoints, since theirhorizontal position on the schedule is indicative of both theireffective time and their entry time. Thus, in the example shown in FIG.41B, at 7:30 AM a user made an RT setpoint entry “i” having atemperature value of 76° F., at 7:40 AM a user made another RT setpointentry “j” having a temperature value of 72° F., at 9:30 AM a user madeanother RT setpoint entry “l” having a temperature value of 72° F., at11:30 AM a user made another RT setpoint entry “m” having a temperaturevalue of 76° F., and so on. On Tuesday, at 10 AM, a user created,through a scheduling interface, an NRT setpoint entry “n” that is totake effect on Tuesdays at 12:00 PM and created an NRT setpoint entry“w” that is to take effect on Tuesdays at 9:00 PM. Subsequently, onTuesday at 4:00 PM, a user created an NRT setpoint entry “h” that is totake effect on Mondays at 9:15 PM and created an NRT setpoint entry “k”that was to take effect on Tuesdays at 9:15 AM. Finally, on Tuesday at 8PM, a user created an NRT setpoint entry “s” that is to take effect onTuesdays at 6:00 PM.

Referring now to step 4010, throughout the 24-hour monitoring period,the intelligent thermostat controls the HVAC system according towhatever current version of the control schedule is in effect as well aswhatever RT setpoint entries are made by the user and whatever NRTsetpoint entries have been made that are causally applicable. The effectof an RT setpoint entry on the current setpoint temperature ismaintained until the next pre-existing setpoint is encountered, until acausally applicable NRT setpoint is encountered, or until a subsequentRT setpoint entry is made. Thus, with reference to FIGS. 41A-41B, onTuesday morning, at 6:45 PM, the current operating setpoint of thethermostat changes to 73° F. due to pre-existing setpoint “b,” then, at7:30 AM, the current operating setpoint changes to 76° F. due to RTsetpoint entry “i,” then, at 7:45 AM, the current operating setpointchanges to 72° F. due to RT setpoint entry “j,” then, at 8:15 AM, thecurrent operating setpoint changes to 65° F. due to pre-existingsetpoint entry “c,” then, at 9:30 AM, the current operating setpointchanges to 72° F. due to RT setpoint entry “l,” then, at 11:30 AM, thecurrent operating setpoint changes to 76° F. due to RT setpoint entry“m,” then at 12:00 PM the current operating setpoint changes to 71° F.due to NRT setpoint entry “n,” then, at 12:15 PM, the current operatingsetpoint changes to 78° F., due to RT setpoint entry “o,” and so forth.At 9:15 AM, there is no change in the current setpoint due to NRTsetpoint entry “k” because it did not yet exist. By contrast, the NRTsetpoint entry “n” is causally applicable because it was entered by theuser at 10 AM that day and took effect at its designated effective timeof 12:00 PM.

According to one optional alternative embodiment, step 4010 can becarried out so that an RT setpoint entry is only effective for a maximumof 2 hours, or other relatively brief interval, as the operatingsetpoint temperature, with the operating setpoint temperature returningto whatever temperature would be specified by the pre-existing setpointson the current schedule or by any causally applicable NRT setpointentries. This optional alternative embodiment is designed to encouragethe user to make more RT setpoint entries during the initial learningperiod so that the learning process can be achieved more quickly. As anadditional optional alternative, the initial schedule, in step 4004, isassigned with relatively low-energy setpoints, as, for example,relatively low-temperature setpoints in winter, such as 62° F., whichgenerally produces a lower-energy control schedule. As yet anotheralternative, during the first few days, instead of reverting topre-existing setpoints after 2 hours, the operating setpoint insteadreverts to a lowest-energy pre-existing setpoint in the schedule.

Referring now to step 4012, at the end of the monitoring period, thestored RT and NRT setpoints are processed with respect to one anotherand the current schedule to generate a modified version, iteration, orrefinement of the schedule, the particular steps for which are shown inFIG. 40B. This processing can be carried out, for example, at 11:50 PMof the learning day, or at some other time near or around midnight. Whenit is determined that the initial learning is not yet complete, in step4014, the modified version of the schedule is used for another day ofinitial learning, in steps 4006-4010, is yet again modified in step4012, and the process continues until initial learning is complete. Wheninitial learning is complete, steady-state learning begins in step 4016,described below with respect to FIGS. 32A-33.

For some implementations, the decision, in step 4014, regarding whetheror not the initial control-schedule learning is complete is based onboth the passage of time and whether there has been a sufficient amountof user behavior to record and process. For one implementation, theinitial learning is considered to be complete only when two days ofinitial learning have passed and there have been ten separate one-hourintervals in which a user has entered an RT or NRT setpoint. Any of avariety of different criteria can be used to determine whether there hasbeen sufficient user interaction to conclude initial learning.

FIG. 40B illustrates steps for processing stored RT and NRT setpointsthat correspond generally to step 4012 of FIG. 40A. In step 4030,setpoint entries having nearby effective times are grouped intoclusters, as illustrated in FIG. 41C. In one implementation, any set oftwo or more setpoint entries for which the effective time of each memberis separated by less than 30 minutes from that of at least one othermember constitutes a single cluster.

In step 4032, each cluster of setpoint entries is processed to generatea single new setpoint that represents the entire cluster in terms ofeffective time and temperature value. This process is directed tosimplifying the schedule while, at the same time, best capturing thetrue intent of the user by virtue of the user's setpoint-entry behavior.While a variety of different approaches, including averaging oftemperature values and effective times of cluster members, can be used,one method for carrying out step 4032, described in more detail in FIG.40C, takes into account the NRT vs. RT status of each setpoint entry,the effective time of each setpoint entry, and the entry time of eachsetpoint entry.

Referring now to FIG. 40C, which corresponds to step 4032 of FIG. 40B, adetermination is made, in step 4060, whether there are any NRT setpointentries in the cluster having an entry time that is later than theearliest effective time in the cluster. When this is the case, then, instep 4064, the cluster is replaced by a single representative setpointwith both the effective time and the temperature value of thelatest-entered NRT setpoint entry. This approach provides deference tothe wishes of the user who has taken the time to specifically enter adesired setpoint temperature for that time. When, in step 4060, thereare no such NRT setpoint entries, then, in step 4062, the cluster isreplaced by a single representative setpoint with an effective time ofthe earliest effective cluster member and a setpoint temperature equalto that of the cluster member having the latest entry time. Thisapproach provides deference to the wishes of the user as expressed inthe immediate-control inputs and existing setpoints.

Referring again to FIG. 40B, in step 4034, the new representativesetpoint that determined in step 4032 is tagged with an “RT” or “NRT”label based on the type of setpoint entry from which the setpoint'stemperature value was assigned. Thus, in accordance with the logic ofFIG. 40C, were an NRT setpoint to have the latest-occurring time ofentry for the cluster, the new setpoint would be tagged as “NRT.” Werean RT setpoint to have the latest-occurring time of entry, the newsetpoint would be tagged as “RT.” In steps 4036-4038, any singularsetpoint entries that are not clustered with other setpoint entries aresimply carried through as new setpoints to the next phase of processing,in step 4040.

Referring to FIGS. 41C-41D, it can be seen that, for the “ij” cluster,which has only RT setpoint entries, the single representative setpoint“ij” is assigned to have the earlier effective time of RT setpoint entry“i” while having the temperature value of the later-entered RT setpointentry “j,” representing an application of step 4062 of FIG. 40C, andthat new setpoint “ij” is assigned an “RT” label in step 4034. It canfurther be seen that, for the “kl” cluster, which has an NRT setpoint“k” with an entry time later than the earliest effective time in thatcluster, the single representative setpoint “kl” is assigned to haveboth the effective time and temperature value of the NRT setpoint entry“k,” representing an application of step 4064 of FIG. 40C, and that newsetpoint “kl” is assigned an “NRT” label in step 4034. For the “mno”cluster, which has an NRT setpoint “n” but with an entry time earlierthan the earliest effective time in that cluster, the singlerepresentative setpoint “mno” is assigned to have the earliest effectivetime of RT setpoint entry “m” while having the temperature value of thelatest-entered setpoint entry “o,” again representing an application ofstep 4062 of FIG. 40C, and that new setpoint “mno” is assigned an “RT”label in step 4034. The remaining results shown in FIG. 41D, all ofwhich are also considered to be new setpoints at this stage, also followfrom the methods of FIGS. 40B-40C.

Referring again to FIG. 40B, step 4040 is next carried out after steps4034 and 4038 and applied to the new setpoints as a group, which areshown in FIG. 41D. In step 4040, any new setpoint having an effectivetime that is 31-60 minutes later than that of any other new setpoint ismoved, in time, to have a new effective time that is 60 minutes laterthat that of the other new setpoint. This is shown in FIG. 41E withrespect to the new setpoint “q,” the effective time of which is beingmoved to 5:00 PM so that it is 60 minutes away from the 4:00 PMeffective time of the new setpoint “p.” In one implementation, thisprocess is only performed a single time based on an instantaneoussnapshot of the schedule at the beginning of step 4040. In other words,there is no iterative cascading effect with respect to these newsetpoint separations. Accordingly, while step 4040 results in a timedistribution of new setpoint effective times that are generallyseparated by at least one hour, some new setpoints having effectivetimes separated by less than one hour may remain. These minor varianceshave been found to be tolerable, and often preferable to deleteriouseffects resulting from cascading the operation to achieve absoluteone-hour separations. Furthermore, these one-hour separations can besuccessfully completed later in the algorithm, after processing againstthe pre-existing schedule setpoints. Other separation intervals may beused in alternative implementations.

Referring to step 4042 of FIG. 40B, consistent with the aggressivepurposes associated with initial learning, the new setpoints that havenow been established for the current learning day are next replicatedacross other days of the week that may be expected to have similarsetpoints, when those new setpoints have been tagged as “RT” setpoints.Preferably, new setpoints tagged as “NRT” are not replicated, since itis likely that the user who created the underlying NRT setpoint entryhas already created similar desired NRT setpoint entries. For someimplementations that have been found to be well suited for the creationof a weekly schedule, a predetermined set of replication rules isapplied. These replication rules depend on which day of the week theinitial learning process was first started. The replication rules areoptimized to take into account the practical schedules of a largepopulation of expected users, for which weekends are often differentlystructured than weekdays, while, at the same time, promoting aggressiveinitial-schedule establishment. For one implementation, the replicationrules set forth in Table 1 are applicable.

TABLE 1 If the First Initial Learning And the Current Learning ThenReplicate New Day was . . . Day is . . . Setpoints Onto . . . Any DayMon-Thu Any Day Mon-Fri All Days Mon-Fri Sat or Sun Sat and Sun FridayFri All 7 Days Sat or Sun Sat and Sun Any Day Mon-Thu All Days Mon-FriSaturday Sat or Sun Sat and Sun Any Day Mon-Fri All Days Mon-Fri SundaySun All 7 Days Mon or Tue All 7 Days Any Day Wed-Fri All Days Mon-FriSat Sat and Sun

FIG. 41F illustrates effects of the replication of the RT-tagged newsetpoints of FIG. 41E, from a Tuesday monitoring period, onto thedisplayed portions of the neighboring days Monday and Wednesday. Thus,for example, the RT-tagged new setpoint “x,” having an effective time of11:00 PM, is replicated as new setpoint “x2” on Monday, and all otherweekdays, and the RT-tagged new setpoint “ij,” having an effective timeof 7:30 AM, is replicated as new setpoint “ij2” on Wednesday and allother weekdays. As per the rules of Table 1, all of the other RT-taggednew setpoints, including “mno,” “p,” “q,” and “u,” are also replicatedacross all other weekdays. Neither of the NRT-tagged new setpoints “kl”or “rst” is replicated. The NRT user setpoint entry “h,” which wasentered on Tuesday by a user who desired it to be effective on Mondays,is not replicated.

Referring now to step 4044 of FIG. 40B, the new setpoints and replicatednew setpoints are overlaid onto the current schedule of pre-existingsetpoints, as illustrated in FIG. 41G, which shows the pre-existingsetpoints encircled and the new setpoints not encircled. In many of thesubsequent steps, the RT-tagged and NRT-tagged new setpoints are treatedthe same, and, when so, the “RT” and “NRT” labels are not used indescribing such steps. In step 4046, a mutual-filtering and/ortime-shifting of the new and pre-existing setpoints is carried outaccording to predetermined filtering rules that are designed tooptimally or near optimally capture the pattern information andpreference information, while also simplifying overall schedulecomplexity. While a variety of different approaches can be used, onemethod for carrying out the objective of step 4046 is described, ingreater detail, in FIG. 40D. Finally, in step 4048, the results of step4046 become the newest version of the current schedule that is eitherfurther modified by another initial learning day or that is used as thestarting schedule in the steady-state learning process.

Referring to FIG. 40D, which sets forth one method for carrying out theprocessing of step 4046 of FIG. 40C, a first type of any new setpointhaving an effective time that is less than one hour later than that of afirst pre-existing setpoint and less than one hour earlier than that ofa second pre-existing setpoint is identified in step 4080. Examples ofsuch new setpoints of the first type are circled in dotted lines in FIG.41G. The steps of FIG. 40D are carried out for the entire weeklongschedule, even though only a portion of that schedule is shown in FIG.41G, for explanatory purposes. In step 4081, any new setpoints of thefirst type are deleted when they have effective times less than one hourearlier than the immediately subsequent pre-existing setpoint and whenthey have a temperature value that is not more than one degree F. awayfrom that of the immediately preceding pre-existing setpoint. Forpurposes of step 4081 and other steps in which a nearness or similarityevaluation between the temperature values of two setpoints isundertaken, the comparison of the setpoint values is carried out withrespect to rounded versions of their respective temperature values, therounding being to the nearest one degree F. or to the nearest 0.5 degreeC., even though the temperature values of the setpoints may bemaintained to a precision of 0.2° F. or 0.1° C. for other operationalpurposes. When using rounding, for example, two setpoint temperatures of77.6° F. and 79.4° F. are considered as 1 degree F. apart when each isfirst rounded to the nearest degree F., and therefore not greater than 1degree F. apart. Likewise, two setpoint temperatures of 20.8° C. and21.7° C. will be considered as 0.5 degree C. apart when each is firstrounded to the nearest 0.5 degree C., and therefore not greater than 0.5degree C. apart. When applied to the example scenario at FIG. 41G, newsetpoint “ij” falls within the purview of the rule in step 4081, andthat new setpoint “ij” is thus deleted, as shown in FIG. 41H.

Subsequent to the deletion of any new setpoints of the first type instep 4081, any new setpoint of the first type that has an effective timethat is within 30 minutes of the immediately subsequent pre-existingsetpoint is identified in step 4082. When such first-type setpoints areidentified, they are moved, later in time, to one hour later than theimmediately preceding pre-existing setpoint, and the immediatelysubsequent pre-existing setpoint is deleted. When applied to the examplescenario at FIG. 41G, new setpoint “ij2” falls within the purview of therule in step 4082 and new setpoint “ij2” is therefore moved, later intime, to one hour from the earlier pre-existing setpoint “f,” with thesubsequent pre-existing setpoint “g” deleted, as shown in FIG. 41H.Subsequently, in step 4084, any new setpoint of the first type that hasan effective time that is within 30 minutes of the immediately precedingpre-existing setpoint there is identified. When such a first-typesetpoint is identified, the setpoint is moved, earlier in time, to onehour earlier than the immediately subsequent pre-existing setpoint andthe immediately preceding pre-existing setpoint is deleted. In step4086, for each remaining new setpoint of the first type that is notsubject to the purview of steps 4082 or 4084, the setpoint temperatureof the immediately preceding pre-existing setpoint is changed to that ofthe new setpoint and that new setpoint is deleted.

In step 4087, any RT-tagged new setpoint that is within one hour of animmediately subsequent pre-existing setpoint and that has a temperaturevalue not greater than one degree F. different from an immediatelypreceding pre-existing setpoint is identified and deleted. In step 4088,for each new setpoint, any pre-existing setpoint that is within one hourof that new setpoint is deleted. Thus, for example, FIG. 41I shows apre-existing setpoint “a” that is less than one hour away from the newsetpoint “x2,” and so the pre-existing setpoint “a” is deleted, in FIG.41J. Likewise, the pre-existing setpoint “d” is less than one hour awayfrom the new setpoint “q,” and so the pre-existing setpoint “d” isdeleted, in FIG. 41J.

In step 4090, starting from the earliest effective setpoint time in theschedule and moving later in time to the latest effective setpoint time,a setpoint is deleted when the setpoint has a temperature value thatdiffers by not more than 1 degree F. or 0.5 degree C. from that of theimmediately preceding setpoint. As discussed above, anchor setpoints, inmany implementations, are not deleted or adjusted as a result ofautomatic schedule learning. For example, FIG. 41K shows the setpoints“mno” and “x” that are each not more than one degree F. from immediatelypreceding setpoints, and so setpoints “mno” and “x” are deleted, in FIG.41L. Finally, in step 4092, when there are any remaining pairs ofsetpoints, new or pre-existing, having effective times that are lessthan one hour apart, the later effective setpoint of each pair isdeleted. The surviving setpoints are then established as members of thecurrent schedule, as indicated in FIG. 41M, all of which are labeled“pre-existing setpoints” for subsequent iterations of the initiallearning process of FIG. 40A or, when that process is complete, forsubsequent application of steady-state learning, described below. Ofcourse, the various time intervals for invoking the above-discussedclustering, resolving, filtering, and shifting operations may vary, inalternative implementations.

FIGS. 42A-42B illustrate steps for steady-state learning. Many of thesame concepts and teachings described above for the initial learningprocess are applicable to steady-state learning, including the trackingof real-time user setpoint entries and non-real time user setpointentries, clustering, resolving, replicating, overlaying, and finalfiltering and shifting.

Certain differences arise between initial and steady state learning, inthat, for the steady-state learning process, there is an attention tothe detection of historical patterns in the setpoint entries, anincreased selectivity in the target days across which the detectedsetpoint patterns are replicated, and other differences. Referring toFIG. 42A, the steady state learning process begins in step 4202, whichcan correspond to the completion of the initial learning process (FIG.40A, step 4016), and which can optionally correspond to a resumption ofsteady-state learning after a user-requested pause in learning. In step4204, a suitable version of the current schedule is accessed. When thesteady-state learning is being invoked immediately following initiallearning, often be the case for a new intelligent-thermostatinstallation, the control schedule is generally the current schedule atthe completion of initial learning.

However, a previously established schedule may be accessed in step 4204,in certain implementations. A plurality of different schedules that werepreviously built up by the intelligent thermostat 3302 over a similarperiod in the preceding year can be stored in the thermostat 3302, or,alternatively, in a cloud server to which it has a network connection.For example, there may be a “January” schedule that was built up overthe preceding January and then stored to memory on January 31. When step4204 is being carried out on January 1 of the following year, thepreviously stored “January” schedule can be accessed. In certainimplementations, the intelligent thermostat 3302 may establish and storeschedules that are applicable for any of a variety of time periods andthen later access those schedules, in step 4204, for use as the nextcurrent schedule. Similar storage and recall methods are applicable forthe historical RT/NRT setpoint entry databases that are discussedfurther below.

In step 4206, a new day of steady-state learning is begun. In step 4208,throughout the day, the intelligent thermostat receives and tracks bothreal-time and non-real time user setpoint entries. In step 4210,throughout the day, the intelligent thermostat proceeds to control anHVAC system according to the current version of the schedule, whateverRT setpoint entries are made by the user, and whatever NRT setpointentries have been made that are causally applicable.

According to one optional alternative embodiment, step 4210 can becarried out so that any RT setpoint entry is effective only for amaximum of 4 hours, after which the operating setpoint temperature isreturned to whatever temperature is specified by the pre-existingsetpoints in the current schedule and/or whatever temperature isspecified by any causally applicable NRT setpoint entries. As anotheralternative, instead of reverting to any pre-existing setpoints after 4hours, the operating setpoint instead reverts to a relatively low energyvalue, such as a lowest pre-existing setpoint in the schedule. Thislow-energy bias operation can be initiated according to a user-settablemode of operation.

At the end of the steady-state learning day, such as at or aroundmidnight, processing steps 4212-4216 are carried out. In step 4212, ahistorical database of RT and NRT user setpoint entries, which mayextend back at least two weeks, is accessed. In step 4214, the day'stracked RT/NRT setpoint entries are processed in conjunction with thehistorical database of RT/NRT setpoint entries and the pre-existingsetpoints in the current schedule to generate a modified version of thecurrent schedule, using steps that are described further below withrespect to FIG. 42B. In step 4216, the day's tracked RT/NRT setpointentries are then added to the historical database for subsequent use inthe next iteration of the method. Notably, in step 4218, whether thereshould be a substitution of the current schedule to something that ismore appropriate and/or preferable is determined, such as for a changeof season, a change of month, or another such change. When a schedulechange is determined to be appropriate, a suitable schedule is accessedin step 4204, before the next iteration. Otherwise, the next iterationis begun in step 4206 using the most recently computed schedule. Incertain implementations, step 4218 is carried out based on direct userinstruction, remote instruction from an automated program running on anassociated cloud server, remote instruction from a utility company,automatically based on the present date and/or current/forecastedweather trends, or based on a combination of one or more of the abovecriteria or other criteria.

Referring to FIG. 42B, which corresponds to step 4214 of FIG. 42A, stepssimilar to those of steps 4030-4040 of FIG. 40B are carried out in orderto cluster, resolve, tag, and adjust the day's tracked RT/NRT setpointentries and historical RT/NRT setpoint entries. In step 4232, allRT-tagged setpoints appearing in the results of step 4232 are identifiedas pattern-candidate setpoints. In step 4234, the current day'spattern-candidate setpoints are compared to historical pattern-candidatesetpoints to detect patterns, such as day-wise or week-wise patterns, ofsimilar effective times and similar setpoint temperatures. In step 4236,for any such patterns detected in step 4234 that include a current-daypattern-candidate setpoint, the current-day pattern-candidate setpointis replicated across all other days in the schedule for which suchpattern may be expected to be applicable. As an example, Table 2illustrates one particularly useful set of pattern-matching rules andassociated setpoint replication rules.

TABLE 2 If Today And the Detected Then Replicate The Was . . . Match isWith . . . Matched Support Onto . . . Tue Yesterday All Days Mon-FriLast Tuesday Tuesdays Only Wed Yesterday All Days Mon-Fri Last WednesdayWednesdays Only Thu Yesterday All Days Mon-Fri Last Thursday ThursdaysOnly Fri Yesterday All Days Mon-Fri Last Friday Fridays Only SatYesterday All 7 Days of Week Last Saturday Saturdays Only Sun YesterdaySaturdays and Sundays Last Sunday Sundays Only Mon Yesterday All 7 Daysof Week Last Monday Mondays Only

For one implementation, in carrying out step 4236, the replicatedsetpoints are assigned the same effective time of day, and the sametemperature value, as the particular current day pattern-candidatesetpoint for which a pattern is detected. In other implementations, thereplicated setpoints can be assigned the effective time of day of thehistorical pattern-candidate setpoint that was involved in the matchand/or the temperature value of that historical pattern-candidatesetpoint. In still other implementations, the replicated setpoints canbe assigned the average effective time of day of the current andhistorical pattern-candidate setpoints that were matched and/or theaverage temperature value of the current and historicalpattern-candidate setpoints that were matched.

In step 4238, the resulting replicated schedule of new setpoints isoverlaid onto the current schedule of pre-existing setpoints. Also, instep 4238, any NRT-tagged setpoints resulting from step 4230 areoverlaid onto the current schedule of pre-existing setpoints. In step4240, the overlaid new and pre-existing setpoints are then mutuallyfiltered and/or shifted in effective time using methods similar to thosediscussed above for step 4046 of FIG. 40B. The results are thenestablished, in step 4242, as the newest version of the currentschedule.

Background Schedule Simulation

FIG. 43A illustrates the intelligent thermostat 102 as installed in asmart home environment 100 having an HVAC system 4399 and a set ofcontrol wires 4398 extending therefrom. The intelligent thermostat 102is, of course, extremely well suited for installation by contractors innew home construction and/or in the context of complete HVAC systemreplacement. However, one alternative key business opportunity leveragedaccording to one embodiment is the marketing and retailing of theintelligent thermostat 102 as a replacement thermostat in an existinghome, wherein the customer (and/or an HVAC professional) disconnectstheir old thermostat from the existing wires 298 and substitutes in theintelligent thermostat 102.

In either case, the intelligent thermostat 102 can advantageously serveas an “inertial wedge” for inserting an entire energy-saving technologyplatform into the home. Simply stated, because most homeownersunderstand and accept the need for home to have a thermostat, even themost curmudgeonly and techno-phobic homeowners will readily accept thesimple, non-intimidating, and easy-to-use intelligent thermostat 102into their homes. Once in the home, of course, the intelligentthermostat 102 will advantageously begin saving energy for a sustainableplanet and saving money for the homeowner, including the curmudgeons.Additionally, however, as homeowners “warm up” to the intelligentthermostat 102 platform and begin to further appreciate its delightfulelegance and seamless operation, they will be more inclined to takeadvantage of its advanced features, and they will furthermore be moreopen and willing to embrace a variety of compatible follow-on productsand services as are described further hereinbelow. This is anadvantageous win-win situation on many fronts, because the planet isbenefiting from the propagation of energy-efficient technology, while atthe same time the manufacturer of the intelligent thermostat and/ortheir authorized business partners can further expand their businessrevenues and prospects.

FIG. 43B illustrates an exemplary diagram of the HVAC system 4399 ofFIG. 43A. HVAC system 4399 provides heating, cooling, ventilation,and/or air handling for an enclosure, such as the smart home environment100 depicted in FIG. 43A. The HVAC system 4399 depicts a forced air typeheating system, although according to other embodiments, other types ofsystems could be used. In heating, heating coils or elements 4342 withinair handler 4340 provide a source of heat using electricity or gas vialine 4336. Cool air is drawn from the enclosure via return air duct 4346through filter 4370 using fan 4338 and is heated by the heating coils orelements 4342. The heated air flows back into the enclosure at one ormore locations through a supply air duct system 4352 and supply airgrills such as grill 4350. In cooling, an outside compressor 4330 passesa gas such as Freon through a set of heat exchanger coils to cool thegas. The gas then goes via line 4332 to the cooling coils 4334 in theair handlers 4340 where it expands, cools and cools the air beingcirculated through the enclosure via fan 4338. According to someembodiments a humidifier 4362 is also provided which moistens the airusing water provided by a water line 4364. Although not shown in FIG.43B, according to some embodiments the HVAC system for the enclosure hasother known components such as dedicated outside vents to pass air toand from the outside, one or more dampers to control airflow within theduct systems, an emergency heating unit, and a dehumidifier. The HVACsystem is selectively actuated via control electronics 4312 thatcommunicate with the intelligent thermostat 102 over control wires 4398.

FIG. 44 illustrates one embodiment of components of a backgroundschedule simulation system 4400. The background schedule simulationsystem 4400 can generate simulation data indicating a simulated energyconsumption and/or a simulated energy cost over a period of time. Thisgenerated energy consumption and/or simulated energy cost can then becompared to the actual energy consumption and/or actual energy cost overthe same period of time. Advantageously, this comparison can be used toincentivize a user, even the most environmentally unfriendly user, toconserve energy by displaying the comparative cost of the inefficientenergy consumption.

The background schedule simulation system 4400 can be a component of thesmart home environment 100, and can be co-located within the intelligentthermostat 102. In some embodiments, the background schedule simulationsystem can be configured to receive inputs from and provide outputs toother components of the smart home environment 100. These inputs can beenvironmental inputs including data relating to an environmentalparameter of a climate controlled enclosure within the smart homeenvironment 100, such as the ambient temperature within a building,including a home, or within a portion of the building, including a room,data relating to environmental parameter of the area outside of theclimate controlled enclosure, such as the ambient temperature outsidethe building or outside a portion of the building, data relating to thehuman occupancy of the climate controlled enclosure including, forexample, whether a human is in the climate controlled enclosure, energyprice information, and energy consumption information.

In some embodiments, the inputs received by the background schedulesimulation system 4400 can include user inputs including a controlschedule such as an HVAC schedule that can include one or severalsetpoints. The setpoints can include a setpoint time and a setpointtemperature, and can thereby define a desired environmental parameter,such as a temperature, for the climate controlled enclosure and a timewhen the setpoint is applicable to the climate controlled enclosure. Insome embodiments, the setpoint time can define: a starting time for theapplicability of the setpoint to the climate controlled enclosure, astarting time and an end time that bound the applicability of thesetpoint to the climate controlled enclosure, and/or a starting time anda duration that bound the applicability of the setpoint to the climatecontrolled enclosure. In some embodiments in which the HVAC schedulecomprises a plurality of setpoints, the setpoint time can define astarting time for the applicability of the setpoint to the climatecontrolled enclosure, and the applicability of the setpoint to theclimate controlled enclosure can last until the setpoint time of adifferent setpoint is reached.

As discussed above, the setpoints can be set when the HVAC schedule iscreated, and the setpoints can be changed while the HVAC schedule isbeing operated, which change can be accomplished via an update which canbe received by the background schedule simulation system 4400. In someembodiments, and as discussed above setpoints can be added, removed,and/or changed, including changing one or both of the setpoint time andthe setpoint temperature. A setpoint can be changed, as discussed above,by the user providing an RT setpoint and/or an NRT setpoint. In someembodiments, a setpoint can be further changed based on the occupancy ofthe climate controlled enclosure. Thus, in some embodiments, a setpointresulting in the consumption of a lesser amount of energy can beimplemented when the climate controlled enclosure is not occupied by ahuman.

Referring again to FIG. 44 the background schedule simulation system4400 can include an HVAC system control 4402. The HVAC system control4402 can control the HVAC system 4399 based on received inputs. In someembodiments, the HVAC system control 4402 can receive the user providedHVAC schedule, hereinafter referred to as the first HVAC schedule, canreceive updates including one or several RT setpoints, one or severalNRT setpoints, and/or an occupancy profile indicating whether theclimate controlled enclosure is occupied by a human, and an inputindicative of an environmental parameter of the climate controlledenclosure including, for example, the ambient temperature in the climatecontrolled enclosure.

The HVAC system control 4402 can generate HVAC system control signals,an actual runtime profile, and/or a record HVAC schedule based on theinputs. The HVAC system control signals can control the operation of theHVAC system, and specifically can comprise the instructions to the HVACsystem 4399 that turn the HVAC system 4399 on and off. The record HVACschedule can be a historical record of the setpoints executed by theHVAC system control 4402. The actual runtime profile can comprise thecollection of the control signals generated by the HVAC system control4402 over the period of time. Thus, the actual runtime profile is ahistoric record of when the HVAC system 4399 was turned on and theduration of the operation of the HVAC system 4399.

The background schedule simulation system 4400 includes a backgroundschedule simulation learning engine 4404. The background schedulesimulation learning engine 4404 can, like the HVAC system control 4402receive inputs relating to and used for controlling the operation of theHVAC system 4399. In some embodiments, the background schedulesimulation learning engine 4404 can receive the first HVAC schedule, andcan receive updates including one or several RT setpoints, one orseveral NRT setpoints, and/or an occupancy profile indicating whetherthe climate controlled enclosure is occupied by a human. In oneparticular embodiment, the background schedule simulation learningengine 4404 can receive the first HVAC schedule and can receive updatesincluding one or several RT setpoints and/or one or several NRTsetpoints.

The background schedule simulation engine 4404 can generate a learnedschedule, hereinafter referred to as the second HVAC schedule. Thesecond HVAC schedule can be generated using the learning processes andalgorithms discussed in greater detail above, and can be used as thebasis for alternate operation of the HVAC system 4399 in the comparisonof actual energy consumption and/or actual energy cost with thesimulated energy consumption and/or simulated energy cost.

The background schedule simulation system 4400 includes a HVAC/structuremodeling engine 4406. The HVAC/structure modeling engine 4406 cangenerate a model of the HVAC system 4399 and the associated climatecontrolled enclosure. The HVAC/structure modeling engine 4406 can use avariety of inputs in generating the model of the HVAC system 4399 in theassociated climate controlled enclosure. In some embodiments, forexample, these inputs can include the actual runtime profile generatedby the HVAC system control 4402, the ambient temperature inside theclimate controlled enclosure, the energy consumption of the HVAC system4399 during operation, and the ambient temperature of the area outsideand surrounding the climate controlled enclosure. This information canbe used to determine the rate of ambient temperature change in theclimate control enclosure when the HVAC system 4399 is not in operationand the rate of ambient temperature change when the HVAC system 4399 isin operation.

In some embodiments, for example, the HVAC/structure modeling engine4406 generates a HVAC/structure model which is a model of a thermalproperty of the HVAC system 4399 and the climate controlled enclosure,which thermal property can relate to the rate of heat transfer to andfrom the climate controlled enclosure and the amount of energy consumedby the HVAC system 4399 to affect an ambient temperature change insidethe climate controlled enclosure. In some embodiments, for example, thisthermal property can include, for example, one or several of a thermalresistance, a thermal conductivity, and/or a heat flux. As used herein,a model of a thermal property of an HVAC system, or thermal propertymodel, includes information sufficient to be able to compute HVAC systemruntime results in view of at least one input driving function. Suchinput driving function can include, for example, an actual orhypothetical HVAC schedule, an actual or hypothetical outsidetemperature profile versus time, actual or hypothetical occupancyinformation versus time, and so forth. The thermal property model caninclude, but is not limited to, a thermal conductivity metric thatreflects the ability of the structure to hold heat/cool, theheating/cooling capacity of the HVAC system, the size of the home insquare feet and/or cubic feet, and so forth. The thermal property modelcan additionally and/or alternatively include specialized parametersthat are specific to one or more abstract models of the HVACcharacteristics of the home that do not necessarily map intoconventional units or metrics such as BTU/hour or square footage, butthat instead map specifically into internal control system parameters ofthose abstract models. For some embodiments, one or more of the thermalmodeling methods described in the commonly assigned U.S. Ser. No.12/881,463 can be used to generate the thermal property model.

The background schedule simulation system 4400 can include a setpointprofile simulator 4408. In some embodiments, for example, the setpointprofile simulator 4408 can generate a setpoint profile comprisingsimulated setpoints simulated to occur during the period of time. Insome embodiments, the setpoint profile simulator 4408 can receive inputsfrom the other components of the background schedule simulation system4400 including, for example, the second HVAC schedule, which wasgenerated by the learning engine 4404, and the occupancy profile. Insome embodiments, for example, the setpoint profile simulator 4408 canmerge the second HVAC schedule and the occupancy profile to identifywhich setpoints would have been affected by the absence of a human fromthe climate controlled enclosure. The setpoint profile simulator 4408can then generate simulated setpoints for times in which no human waspresent in the climate controlled enclosure. These setpoints can becombined with the setpoints of the second HVAC schedule simulated asoccurring during times in which a human was present in the climatecontrolled enclosure to create a setpoint profile, which represents thesetpoints that would have been used in the operation the HVAC system4399 during the period of time if the HVAC system 4399 had operatedaccording to the second HVAC schedule.

The background schedule simulation system 4400 includes a cost simulator4410. The cost simulator 4410 can be configured to receive inputs fromother components of the background schedule simulation system 4400 andenergy price information, which information can comprise actual energyprice information for the time period or estimated price information forthe time period. The cost simulator 4410 can use these inputs and theenergy price information to generate an estimated cost of operating theHVAC system 4399 according to the provided inputs and the energy priceinformation. In one embodiment, for example, the cost simulator 4410 canreceive an input identifying the rate of energy consumption by the HVACsystem 4399 when the HVAC system 4399 is being operated, a runtimeprofile which can be, for example, the actual runtime profile and/or asimulated runtime profile, and energy price information. In someembodiments, when the cost simulator 4410 uses the actual runtimeprofile, the actual energy consumption and/or the actual energy cost canbe calculated, and in embodiments when the cost simulator 4410 uses thesimulated runtime profile, the simulated energy consumption and/or thesimulated energy cost can be calculated.

FIG. 45 illustrates one embodiment of a process 4500 for backgroundschedule simulation. This process 4500 can be performed with thecomponents of the background schedule simulation system 4400 discussedabove. The process 4500 begins at decision state 4502 wherein it isdetermined if learning is enabled. In some embodiments, for example, thedetermination of whether learning is enabled can include querying theuser as to whether to enable learning, and receiving an input from theuser indicating whether to enable learning. If learning is enabled, theprocess proceeds to block 4504 and generates and executes the learnedschedule as discussed above.

Returning again to decision state 4502, if learning is not enabled, theprocess 4500 proceeds to block 4506 wherein the first HVAC schedule isreceived. In some embodiments, for example, the first HVAC schedule canbe received from the user and can, for example, be received by theintelligent thermostat 102 and provided to the background schedulesimulation system 4400.

After the first HVAC schedule has been received, the process 4500proceeds to block 4508 wherein ambient temperature data for the climatecontrolled enclosure is received. In some embodiments, for example, theambient temperature data for the climate controlled enclosure can bemeasured by a component of the smart home environment 100 including, forexample, a sensor located in the intelligent thermostat 102. Thisambient temperature data for the climate control enclosure can beprovided to the background schedule simulation system 4400.

After the ambient temperature data for the climate controlled enclosurehas been received, the process 4500 proceeds to decision state 4510wherein it is determined if an update has been received. In someembodiments this determination can include determining whether the userhas provided an RT setpoint and/or an NRT setpoint, or if the occupancystatus of the climate controlled enclosure has changed.

If it is determined that an update has been received, then the process4500 proceeds to block 4512 wherein the first and second HVAC schedulesare updated. In some embodiments, for example, the first HVAC schedulecan be updated by the HVAC system control 4402 to reflect the receivedupdates. In some embodiments, for example, the second HVAC schedule canbe updated by the learning engine 4404 based on the learning algorithmsand processes discussed above in connection with the received updates.Although both the first HVAC schedule and the second HVAC schedule areupdated based on the updates as the second HVAC schedule is also basedon the above discussed learning algorithms and learning processes, thefirst HVAC schedule and the second HVAC schedule can differ.

After the first and second HVAC schedules have been updated, andreturning again to decision state 4510, if it is determined that noupdate has been received, the process 4500 proceeds to block 4514wherein a control signal is generated. The control signal can begenerated by the HVAC system control 4402 based on the first HVACschedule and any update thereto. As discussed above, the control signalcan control the operation of the HVAC system 4399 including, forexample, when the HVAC system 4399 is operating and when the HVAC system4399 is not operating.

After the control signal is generated, the process 4500 proceeds todecision state 4516 wherein it is determined if the time period hasended. As discussed above, in some embodiments the background schedulesimulation system 4400 is configured to compare actual energyconsumption and/or actual energy cost to simulated energy consumptionand/or simulated energy cost over a period of time. In some embodiments,the background schedule simulation engine 4400 can be configured todetermine if this period of time has ended. If the period of time hasnot ended, the process 4500 returns to block 4508 and continues asdescribed above.

If it is determined that the period of time has ended, the process 4500proceeds to block 4518 wherein the actual runtime profile is compiled.In some embodiments, for example, the actual runtime profile cancomprise the collection of control signals generated by the HVAC systemcontrol 4402 over the period of time.

After the actual runtime profile is compiled, the process 4500 proceedsto block 4520 wherein HVAC consumption data is received. In someembodiments, for example, the HVAC consumption data can comprise dataindicating the amount of energy consumed by the HVAC system 4399 perlength of time that the HVAC system 4399 is in operation. In someembodiments, for example, this information can be received from acomponent of the smart home environment 100 including, for example, theHVAC system 4399, a smart meter, or any other device or sourcecontaining this information.

After the HVAC consumption data has been received, the process 4500proceeds to block 4522 wherein the HVAC/structure model is generated. Insome embodiments, the HVAC/structure model can be generated by theHVAC/structure modeling engine 4406. As discussed above, in someembodiments, the HVAC/structure modeling engine 4406 can generate amodel of the HVAC system 4399 and the associated climate controlledenclosure which can be a model of one or several thermal properties ofthe HVAC system 4399 and the associated climate controlled enclosure.

After the HVAC/structure model has been generated, the process proceedsto block 4524 wherein a simulated setpoint profile is generated. In someembodiments, for example, the simulated setpoint profile can begenerated by the setpoint profile simulator 4408. In some embodiments,and as discussed above, the simulated setpoint profile can be generatedbased on the second HVAC schedule and the occupancy profile, and caninclude the setpoints that would have controlled the operation of theHVAC system 4399 during the period of time if the HVAC system 4399 hadoperated according to the second HVAC schedule for that time.

After the simulated setpoint profile has been generated, the process4500 proceeds to block 4526 wherein the simulated runtime profile isgenerated. In some embodiments, for example, the simulated runtimeprofile can be generated based on the HVAC/structure model generated inblock 4522, based on the simulated setpoint profile generated in block4524, and/or based on the ambient temperature outside the climatecontrolled enclosure. In some embodiments, for example, the simulatedruntime profile can be generated by the HVAC/structure modeling engine4408.

After the simulated runtime profile has been generated, the process 4500proceeds to block 4528 wherein energy price information is received. Insome embodiments, for example, the energy price information can beactual price information for the period of time or estimated priceinformation for the period of time. In some embodiments, for example,the energy price information can be stored in one of the components ofthe background schedule simulation system 4400 and/or in the intelligentthermostat 102 or other component of the smart home environment 100. Insome embodiments, for example, the energy price information can bereceived from a source outside of the smart home environment 100 suchas, for example, the energy provider and/or the user.

After the energy price information has been received, the process 4500proceeds to block 4530 wherein energy cost information is generated. Insome embodiments, for example, the energy cost information can begenerated by the cost simulator 4410 and can be based on the energyprice information, the HVAC consumption data received in block 4520, andone of the actual runtime profile and the simulated runtime profile. Insome embodiments, for example, the cost simulator 4410 can generate anactual energy cost based on the actual runtime profile and the simulatedenergy cost based on the simulated runtime profile.

After the energy cost information is generated, the process 4500proceeds to decision state 4532 wherein it is determined if the actualenergy cost is greater than the simulated energy cost. In someembodiments, for example, the determination of whether the actual energycost is greater than the simulated energy cost can be made by the costsimulator 4410 of the background schedule simulation system 4400.

If it is determined that the actual cost of energy consumed is less thanthe simulated cost of energy consumed, then the process 4500 returns toblock 4508 and proceeds as discussed above. If it is determined that theactual cost of energy consumption is greater than the simulated cost ofenergy consumption, then the process 4500 proceeds to block 4534 whereinan indicator of lost savings is provided. In some embodiments, forexample, this indicator of lost savings can identify the length of theperiod of time and the amount of money and/or energy that could havebeen saved during that period of time if the user had elected togenerate and execute a learned schedule. In some embodiments thisindicator can include a statement such as, “Hey, you could have saved Xdollars!” and a question asking whether the user would like to startsaving money or energy.

After the indicator of lost savings has been provided, the process 4500proceeds to decision state 4536 wherein it is determined if learning isenabled. In some embodiments, for example, the determination of whetherlearning is enabled can include querying the user as to whether toenable learning, which can include querying the user as to whether hewould like to save money and/or energy, and receiving an input from theuser indicating whether to enable learning and/or whether to save moneyand/or energy. If learning is not enabled, then the process 4500proceeds to block 4508.

If learning is enabled, then the process 4500 proceeds to block 4538wherein the HVAC control system 4402 switches to operation according tothe second HVAC schedule. In some embodiments, for example, operationaccording to an HVAC schedule can comprise operating the HVAC system4399 based only on the setpoints in the HVAC schedule, and in someembodiments, operation according to an HVAC schedule can compriseoperating the HVAC system 4399 based on the setpoints in the HVACschedule and on other data. In some embodiments, for example, operatingthe HVAC system 4399 according to the HVAC schedule can includeoperating the HVAC system 4399 based on HVAC schedule setpoints and anyreceived updates. A person of skill in the art will recognize that thephrase “according to” is not narrowly defined to mean only operation ofthe HVAC system 4399 in exact compliance with each HVAC schedulesetpoint, but includes operation of the HVAC system 4399 in compliancewith some or all of the HVAC schedule setpoints and/or in compliancewith any other received data.

In some embodiments, for example, the switch from operating the HVACsystem 4399 according to the first HVAC schedule to according to thesecond HVAC schedule can be performed over a transition period. In someembodiments, for example, this transition period can be short including,for example, a length of time such as a nanosecond or a microsecond, andin some embodiments, for example, this transition period can be extendedto a length of time such as, for example, several weeks or months, andmay specifically comprise two weeks. In some embodiments in which thetransition period is extended, the switch from operating the HVAC system4399 according to the first HVAC schedule to according to the secondHVAC schedule can be gradually performed during the transition period.Advantageously, the gradual switch between HVAC schedules can decreasethe ability of the user to notice the transition and increase thelikelihood of the user remaining with the learned, second HVAC schedule.

FIGS. 46 to 48 illustrate examples of remote graphical user interfacedisplays presented to the user on the data appliance for managing theirbackground schedule simulation system and/or otherwise interacting withtheir smart home environment including the background schedulesimulation system. For the embodiment in FIG. 46, the user interfacedisplay is provided directly by a designated one of the user'sintelligent thermostats 102. For the embodiment shown in FIG. 47, theuser interface display is provided on a portable data device such as asmart phone including an iPhone. For the embodiment shown in FIG. 48,the user interface display is provided in a conventional browser window.

In one embodiment, and as seen in FIGS. 46 to 48, the user interfacedisplay can display a message indicating the amount of money that couldhave been saved over the period of time. As specifically seen in FIGS.46 to 48 this message states “You could have saved $52.00 in the pastthree months! Do you want to start saving now?” which message isfollowed by a first option identified with the phrase “Start Saving” anda second option identified by the phrase “Not Now.” In some embodiments,the first option identified with the phrase “Start Saving” cancorrespond to a user decision to enable learning and/or to operate theHVAC system 4399 according to the second HVAC schedule, and the secondoption identified with the phrase “Not Now” can correspond to a userdecision to not enable learning and/or to operate the HVAC system 4399according to the first HVAC schedule. In one embodiment, the userinterface can default to the first option so as to bias the user towardsenabling learning. In some alternative embodiments, the phrase “Not Now”could be replaced by more stern or even pejorative language, such as“Continue Losing” or “Keep Losing,” which may turn out to be moreeffective in driving the behaviors of some groups or sub-groups of usersin certain environments or scenarios.

In the embodiment depicted in FIG. 46, the user interface allows theuser to enter “ring rotation” and “inward click” user inputs to selectbetween the first and second options. In the embodiment depicted inFIGS. 47 and 48, the user interface allows the user to select betweenthe first and second options by, for example, touchscreen, mouseclick,keyboard, keypad, or any other user input.

Although the present invention has been described in terms of particularexamples, it is not intended that the invention be limited to theseexamples. Modifications within the spirit of the invention will beapparent to those skilled in the art. For example, as discussed above,automated control-schedule learning may be employed in a wide variety ofdifferent types of intelligent controllers in order to learn one or moreschedules that may span period of time from milliseconds to years.Intelligent-controller logic may include logic-circuit implementations,firmware, and computer-instruction-based routine and programimplementations, all of which may vary depending on the selected valuesof a wide variety of different implementation and design parameters,including programming language, modular organization, hardware platform,data structures, control structures, and many other such design andimplementation parameters. As discussed above, the steady-state learningmode follows aggressive learning may include multiple different phases,with the intelligent controller generally becoming increasinglyconservative, with regard to schedule modification, with later phases.Automated-control-schedule learning may be carried out within anindividual intelligent controller, may be carried out in distributedfashion among multiple controllers, may be carried out in distributedfashion among one or more intelligent controllers and remote computingfacilities, and may be carried out primarily in remote computingfacilities interconnected with intelligent controllers. By way of stillfurther example, although automated thermal modeling (i.e., withoutrequiring user assistance) is used in one or more embodiments describedabove as a basis for acquiring a home thermal model that enables thesubject background schedule simulations, the scope of the presentteachings is not so limited. Thus, in other embodiments, the user may berequested to provide certain data points that may assist in the buildingof a thermal model, such as may be implemented using one or more methodsof U.S. Ser. No. 13/440,910, supra, in which a persistent backgroundapplication queries the user during periods of “downtime” or “idle”computing activity about their HVAC environment.

It is appreciated that the previous description of the disclosedexamples is provided to enable any person skilled in the art to make oruse the present disclosure. Various modifications to these examples willbe readily apparent to those skilled in the art, and the genericprinciples defined herein may be applied to other examples withoutdeparting from the spirit or scope of the disclosure. Thus, the presentdisclosure is not intended to be limited to the examples shown hereinbut is to be accorded the widest scope consistent with the principlesand novel features disclosed herein.

What is claimed is:
 1. A method for promoting energy efficiency inassociation with an HVAC system of an enclosure, the method comprising:operating, the HVAC system of the enclosure according to a first HVACschedule over a time period; generating a second HVAC schedulerepresentative of what would have been generated by an automatedschedule learning algorithm operating over the time period; simulatingthe second HVAC schedule using a thermal model of the enclosure for thetime period; generating information representative of an energy costdifference between an actual cost of operating the HVAC system accordingto the first HVAC schedule over the time period and a hypothetical costof operating the HVAC system according to the second HVAC schedule overthe time period based on simulating the second HVAC schedule using thethermal model of the enclosure for the time period; and outputting theinformation representative of the energy cost difference via a userinterface.
 2. The method of claim 1, wherein: the second HVAC scheduleis generated in the background based on the first HVAC schedule and theautomated schedule learning algorithm, the second HVAC schedule and thefirst HVAC schedule exist simultaneously, and the second HVAC scheduleis maintained distinct from the first HVAC schedule.
 3. The method ofclaim 1, wherein the information representative of the energy costdifference is based on a runtime of the HVAC system during the timeperiod according to the first HVAC schedule and the second HVACschedule.
 4. The method of claim 3, wherein the informationrepresentative of the energy cost difference is further based on timesat which the HVAC system was operating during the time period accordingto the first HVAC schedule and the second HVAC schedule.
 5. The methodof claim 1, wherein the information representative of the energy costdifference is representative of a difference in energy consumptionbetween the first HVAC schedule and the second HVAC schedule.
 6. Themethod of claim 1, wherein the user interface is displayed remote from anetwork-connected thermostat.
 7. The method of claim 1, whereingeneration and simulation of the second HVAC schedule are performed by anetwork-connected thermostat.
 8. The method of claim 1, whereingeneration and simulation of the second HVAC schedule are performed by acloud-based remote server system that communicates via a network with anetwork-connected thermostat.
 9. The method of claim 1, furthercomprising: generating a first runtime profile indicating time intervalsfor which the HVAC system was actively heating or cooling over the timeperiod, and wherein generating information representative of the energycost difference comprises: receiving outside temperature data for thetime period, the outside temperature data corresponding to ambienttemperature in an area outside the enclosure; processing a measuredambient temperature in the enclosure over the time period in conjunctionwith the outside temperature data over the time period and the firstruntime profile to generate the thermal model of the enclosure; andprocessing the outside temperature data in conjunction with the thermalmodel of the enclosure and the second HVAC schedule to generate a secondruntime profile.
 10. A system for promoting energy efficiency inassociation with an HVAC system of an enclosure, the system comprising:a network-enabled thermostat, wherein the HVAC system is controlled bythe network-connected thermostat associated with a user interface; acloud-based server system that is in communication with thenetwork-enabled thermostat, wherein the network-enabled thermostat andthe cloud-based server system function as part of the system to:operate, over a time period, the HVAC system according to a first HVACschedule; process the first HVAC schedule to generate a second HVACschedule representative of what would have been generated by anautomated schedule learning algorithm operating over the time period;simulate the second HVAC schedule using a thermal model of theenclosure; generate information representative of an energy costdifference between an actual cost of operating the HVAC system accordingto the first HVAC schedule over the time period and a hypothetical costof operating the HVAC system according to the second HVAC schedule overthe time period based on simulating the second HVAC schedule using thethermal model of the enclosure; and output the informationrepresentative of the energy cost difference via the user interface. 11.The system of claim 10, wherein the information representative of theenergy cost difference is based on a runtime of the HVAC system duringthe time period according the first HVAC schedule and the second HVACschedule.
 12. The system of claim 11, wherein the informationrepresentative of the energy cost difference is further based on timesat which the HVAC system was operating during the time period accordingthe first HVAC schedule and the second HVAC schedule.
 13. The system ofclaim 10, wherein the information representative of the energy costdifference is representative of a difference in energy consumption. 14.The system of claim 10, wherein the user interface is displayed remotefrom the network-connected thermostat on a mobile user device or isdisplayed by the network-enabled thermostat.
 15. The system of claim 10,wherein the network-enabled thermostat of the system generates andsimulates the second HVAC schedule.
 16. The system of claim 10, whereinthe cloud-based server system generates and simulates the second HVACschedule.
 17. The system of claim 10, wherein the network-enabledthermostat and the cloud-based server system function as part of thesystem to: generate a first runtime profile indicating time intervalsfor which the HVAC system was actively heating or cooling over the timeperiod, and wherein generating information representative of the energycost difference comprises: receive outside temperature data for the timeperiod, the outside temperature data corresponding to ambienttemperature in an area outside the enclosure; process an actual ambienttemperature in the enclosure over the time period in conjunction withthe outside temperature data over the time period and the first runtimeprofile to generate the thermal model of the enclosure; and process theoutside temperature data in conjunction with the thermal model of theenclosure and the second HVAC schedule to generate a second runtimeprofile.
 18. A non-transitory processor-readable medium comprisingprocessor-readable instructions that cause one or more processors to:operate, over a time period, an HVAC system according to a first HVACschedule for an enclosure; process the first HVAC schedule to generate asecond HVAC schedule representative of what would have been generated byan automated schedule learning algorithm operating over the time period,wherein: the second HVAC schedule is generated in the background basedon the first HVAC schedule and the automated schedule learningalgorithm, the second HVAC schedule and the first HVAC schedule existsimultaneously, and the second HVAC schedule is maintained distinct fromthe first HVAC schedule; simulate the second HVAC schedule using athermal model of the enclosure; generate information representative ofan energy cost difference between an actual cost of operating the HVACsystem according to the first HVAC schedule over the time period and ahypothetical cost of operating the HVAC system according to the secondHVAC schedule over the time period based on simulating the second HVACschedule using the thermal model of the enclosure; and output theinformation representative of the energy cost difference via a userinterface.
 19. The non-transitory processor-readable medium of claim 18,wherein the information representative of the energy cost difference isbased on a runtime of the HVAC system during the time period.
 20. Thenon-transitory processor-readable medium of claim 18, wherein theprocessor-readable instructions of the non-transitory processor-readablemedium are executed by a cloud-based server system remote from the HVACsystem.